[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:11:26 dist-test-slave-lgr5 krb5kdc[17103](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:11:26 dist-test-slave-lgr5 krb5kdc[17103](info): set up 2 sockets
Aug 06 21:11:26 dist-test-slave-lgr5 krb5kdc[17103](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260806 21:11:28.075614 17096 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Aug 06 21:11:28 dist-test-slave-lgr5 krb5kdc[17103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:11:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:11:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.178.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
I20260806 21:11:28.211748 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:36645
--webserver_interface=127.16.178.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:46781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:36645
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:11:28.358806 17120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:28.359094 17120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:28.359153 17120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:28.363153 17120 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:11:28.363220 17120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:28.363247 17120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:28.363268 17120 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:11:28.363289 17120 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:11:28.369081 17120 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:46781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:36645
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:36645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.178.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17120
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:28.370751 17120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:28.372112 17120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:28.386543 17131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:28.387076 17129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:28.387162 17120 server_base.cc:1034] running on GCE node
W20260806 21:11:28.387316 17126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:28.388106 17120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:28.390054 17120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:28.391227 17120 hybrid_clock.cc:648] HybridClock initialized: now 1786050688391209 us; error 57 us; skew 500 ppm
Aug 06 21:11:28 dist-test-slave-lgr5 krb5kdc[17103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050688, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:28.394222 17120 init.cc:377] Logged in from keytab as kudu/127.16.178.62@KRBTEST.COM (short username kudu)
I20260806 21:11:28.395491 17120 webserver.cc:460] Webserver started at http://127.16.178.62:34101/ using document root <none> and password file <none>
I20260806 21:11:28.396559 17120 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:28.396637 17120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:28.397006 17120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:28.400192 17120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "609d298d383e4fb0821044ec1cffb3ee"
format_stamp: "Formatted at 2026-08-06 21:11:28 on dist-test-slave-lgr5"
server_key: "f13156c119e99f316f8cc9b92d341447"
server_key_iv: "51be48c7e075e67c9d993a7dca2b43ff"
server_key_version: "encryptionkey@0"
I20260806 21:11:28.400889 17120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "609d298d383e4fb0821044ec1cffb3ee"
format_stamp: "Formatted at 2026-08-06 21:11:28 on dist-test-slave-lgr5"
server_key: "f13156c119e99f316f8cc9b92d341447"
server_key_iv: "51be48c7e075e67c9d993a7dca2b43ff"
server_key_version: "encryptionkey@0"
I20260806 21:11:28.407002 17120 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 21:11:28.411049 17145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:28.412879 17120 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 21:11:28.413098 17120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "609d298d383e4fb0821044ec1cffb3ee"
format_stamp: "Formatted at 2026-08-06 21:11:28 on dist-test-slave-lgr5"
server_key: "f13156c119e99f316f8cc9b92d341447"
server_key_iv: "51be48c7e075e67c9d993a7dca2b43ff"
server_key_version: "encryptionkey@0"
I20260806 21:11:28.413231 17120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:28.433629 17120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:28.437316 17120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:28.437526 17120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:28.447321 17120 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.62:36645
I20260806 21:11:28.447356 17229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.62:36645 every 8 connection(s)
I20260806 21:11:28.448755 17120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:11:28.450789 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 17120
I20260806 21:11:28.450934 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:11:28.452474 17096 external_mini_cluster.cc:1428] Setting key db1b7ceb33c3b51b45a6e393071e3e6d
I20260806 21:11:28.454648 17230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:28.464514 17230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 609d298d383e4fb0821044ec1cffb3ee: Bootstrap starting.
I20260806 21:11:28.468689 17230 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 609d298d383e4fb0821044ec1cffb3ee: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:28.470081 17230 log.cc:824] T 00000000000000000000000000000000 P 609d298d383e4fb0821044ec1cffb3ee: Log is configured to *not* fsync() on all Append() calls
Aug 06 21:11:28 dist-test-slave-lgr5 krb5kdc[17103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:11:28.473392 17230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 609d298d383e4fb0821044ec1cffb3ee: No bootstrap required, opened a new log
I20260806 21:11:28.479458 17233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:28.459418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38208 (local address 127.16.178.62:36645)
0806 21:11:28.459947 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:28.459967 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:28.460069 (+ 102us) server_negotiation.cc:407] Connection header received
0806 21:11:28.462840 (+ 2771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:28.462857 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:28.463266 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:28.463572 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:28.466247 (+ 2675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:28.467301 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:28.468531 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:28.469015 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:28.469169 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:28.473606 (+ 4437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:28.473657 (+ 51us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:28.473673 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:28.473703 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:28.475421 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:28.476142 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:28.476147 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:28.476151 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:28.476265 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:28.476665 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:28.476667 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:28.476668 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:28.477203 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:28.477363 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:28.478095 (+ 732us) server_negotiation.cc:299] Negotiation successful
0806 21:11:28.478363 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":153,"threads_started":1}
I20260806 21:11:28.479349 17230 raft_consensus.cc:348] T 00000000000000000000000000000000 P 609d298d383e4fb0821044ec1cffb3ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "609d298d383e4fb0821044ec1cffb3ee" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 36645 } }
I20260806 21:11:28.479718 17230 raft_consensus.cc:374] T 00000000000000000000000000000000 P 609d298d383e4fb0821044ec1cffb3ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:28.479817 17230 raft_consensus.cc:729] T 00000000000000000000000000000000 P 609d298d383e4fb0821044ec1cffb3ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 609d298d383e4fb0821044ec1cffb3ee, State: Initialized, Role: FOLLOWER
I20260806 21:11:28.480743 17230 consensus_queue.cc:260] T 00000000000000000000000000000000 P 609d298d383e4fb0821044ec1cffb3ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "609d298d383e4fb0821044ec1cffb3ee" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 36645 } }
I20260806 21:11:28.481017 17230 raft_consensus.cc:388] T 00000000000000000000000000000000 P 609d298d383e4fb0821044ec1cffb3ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:11:28.481159 17230 raft_consensus.cc:482] T 00000000000000000000000000000000 P 609d298d383e4fb0821044ec1cffb3ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:11:28.481333 17230 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 609d298d383e4fb0821044ec1cffb3ee [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:28.482748 17230 raft_consensus.cc:504] T 00000000000000000000000000000000 P 609d298d383e4fb0821044ec1cffb3ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "609d298d383e4fb0821044ec1cffb3ee" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 36645 } }
I20260806 21:11:28.483397 17230 leader_election.cc:302] T 00000000000000000000000000000000 P 609d298d383e4fb0821044ec1cffb3ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 609d298d383e4fb0821044ec1cffb3ee; no voters:
I20260806 21:11:28.483811 17230 leader_election.cc:288] T 00000000000000000000000000000000 P 609d298d383e4fb0821044ec1cffb3ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:11:28.483953 17236 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 609d298d383e4fb0821044ec1cffb3ee [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:11:28.484237 17236 raft_consensus.cc:686] T 00000000000000000000000000000000 P 609d298d383e4fb0821044ec1cffb3ee [term 1 LEADER]: Becoming Leader. State: Replica: 609d298d383e4fb0821044ec1cffb3ee, State: Running, Role: LEADER
I20260806 21:11:28.484704 17236 consensus_queue.cc:237] T 00000000000000000000000000000000 P 609d298d383e4fb0821044ec1cffb3ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "609d298d383e4fb0821044ec1cffb3ee" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 36645 } }
I20260806 21:11:28.485606 17230 sys_catalog.cc:564] T 00000000000000000000000000000000 P 609d298d383e4fb0821044ec1cffb3ee [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:11:28.487411 17238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 609d298d383e4fb0821044ec1cffb3ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "609d298d383e4fb0821044ec1cffb3ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "609d298d383e4fb0821044ec1cffb3ee" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 36645 } } }
I20260806 21:11:28.487548 17238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 609d298d383e4fb0821044ec1cffb3ee [sys.catalog]: This master's current role is: LEADER
I20260806 21:11:28.487906 17239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 609d298d383e4fb0821044ec1cffb3ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 609d298d383e4fb0821044ec1cffb3ee. Latest consensus state: current_term: 1 leader_uuid: "609d298d383e4fb0821044ec1cffb3ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "609d298d383e4fb0821044ec1cffb3ee" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 36645 } } }
I20260806 21:11:28.488018 17239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 609d298d383e4fb0821044ec1cffb3ee [sys.catalog]: This master's current role is: LEADER
W20260806 21:11:28.492815 17264 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 609d298d383e4fb0821044ec1cffb3ee: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 21:11:28.492945 17264 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 21:11:28.493006 17259 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:11:28.494015 17259 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:11:28.501874 17259 catalog_manager.cc:1347] Generated new cluster ID: f93852db7b034ebfbd351017befdd22a
I20260806 21:11:28.501976 17259 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:11:28.509090 17259 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:11:28.510098 17259 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:11:28.519889 17259 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 609d298d383e4fb0821044ec1cffb3ee: Generated new TSK 0
I20260806 21:11:28.520958 17259 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.178.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab.
Entry for principal kudu/127.16.178.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab.
I20260806 21:11:28.552192 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.1:0
--local_ip_for_outbound_sockets=127.16.178.1
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:36645
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab
--principal=kudu/127.16.178.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:46781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:11:28.670754 17275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:28.671048 17275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:28.671092 17275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:28.675175 17275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:28.675259 17275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:28.675331 17275 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.1
I20260806 21:11:28.681152 17275 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:46781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab
--principal=kudu/127.16.178.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:36645
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17275
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.1
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:28.682549 17275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:28.683667 17275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:28.694773 17280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:28.694857 17283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:28.694811 17289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:28.694986 17275 server_base.cc:1034] running on GCE node
I20260806 21:11:28.695444 17275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:28.696144 17275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:28.697304 17275 hybrid_clock.cc:648] HybridClock initialized: now 1786050688697279 us; error 43 us; skew 500 ppm
Aug 06 21:11:28 dist-test-slave-lgr5 krb5kdc[17103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050688, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:28.700167 17275 init.cc:377] Logged in from keytab as kudu/127.16.178.1@KRBTEST.COM (short username kudu)
I20260806 21:11:28.701133 17275 webserver.cc:460] Webserver started at http://127.16.178.1:37945/ using document root <none> and password file <none>
I20260806 21:11:28.701737 17275 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:28.701817 17275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:28.702029 17275 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:28.704273 17275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-0/data/instance:
uuid: "332863ed73d049bbbe90e1590f82fa2c"
format_stamp: "Formatted at 2026-08-06 21:11:28 on dist-test-slave-lgr5"
server_key: "e5b6e5e492c0924a94a9b8fd8271d5f1"
server_key_iv: "f94039c037b27c30dc7b6ecf6a0944fd"
server_key_version: "encryptionkey@0"
I20260806 21:11:28.704916 17275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance:
uuid: "332863ed73d049bbbe90e1590f82fa2c"
format_stamp: "Formatted at 2026-08-06 21:11:28 on dist-test-slave-lgr5"
server_key: "e5b6e5e492c0924a94a9b8fd8271d5f1"
server_key_iv: "f94039c037b27c30dc7b6ecf6a0944fd"
server_key_version: "encryptionkey@0"
I20260806 21:11:28.708922 17275 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:11:28.711431 17304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:28.712858 17275 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:11:28.712991 17275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-0/wal
uuid: "332863ed73d049bbbe90e1590f82fa2c"
format_stamp: "Formatted at 2026-08-06 21:11:28 on dist-test-slave-lgr5"
server_key: "e5b6e5e492c0924a94a9b8fd8271d5f1"
server_key_iv: "f94039c037b27c30dc7b6ecf6a0944fd"
server_key_version: "encryptionkey@0"
I20260806 21:11:28.713114 17275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:28.734375 17275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:28.737808 17275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:28.738009 17275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:28.738760 17275 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:11:28.740043 17275 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:11:28.740123 17275 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:28.740183 17275 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:11:28.740211 17275 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:28.751231 17275 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.1:42611
I20260806 21:11:28.751259 17476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.1:42611 every 8 connection(s)
I20260806 21:11:28.752221 17275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
I20260806 21:11:28.759847 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 17275
I20260806 21:11:28.759974 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance
I20260806 21:11:28.760460 17096 external_mini_cluster.cc:1428] Setting key cf9ccfceb8eab860be8392d7a85bffdb
Aug 06 21:11:28 dist-test-slave-lgr5 krb5kdc[17103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050688, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:11:28.768723 17233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:28.754634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:52041 (local address 127.16.178.62:36645)
0806 21:11:28.754790 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:28.754794 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:28.755685 (+ 891us) server_negotiation.cc:407] Connection header received
0806 21:11:28.757239 (+ 1554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:28.757242 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:28.757303 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:28.757402 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:28.759102 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:28.759864 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:28.761514 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:28.761776 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:28.761865 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:28.765151 (+ 3286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:28.765155 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:28.765157 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:28.765188 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:28.766503 (+ 1315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:28.767063 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:28.767067 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:28.767068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:28.767149 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:28.767496 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:28.767499 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:28.767501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:28.767699 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:28.767787 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:28.768403 (+ 616us) server_negotiation.cc:299] Negotiation successful
0806 21:11:28.768549 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 21:11:28.769616 17481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:28.755014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:36645 (local address 127.16.178.1:52041)
0806 21:11:28.755529 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:28.755575 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:28.756986 (+ 1411us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:28.757625 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:28.757635 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:28.758171 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:28.758918 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:28.758943 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:28.760017 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:28.760021 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:28.761381 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:28.761398 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:28.761955 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:28.761959 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:28.762158 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:28.762844 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:28.762868 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:28.764928 (+ 2060us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:28.766634 (+ 1706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:28.766641 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:28.766662 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:28.766947 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:28.767237 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:28.767240 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:28.767242 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:28.767394 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:28.767802 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:28.767810 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:28.768140 (+ 330us) client_negotiation.cc:769] Sending connection context
0806 21:11:28.768329 (+ 189us) client_negotiation.cc:240] Negotiation successful
0806 21:11:28.768707 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":122,"threads_started":1}
I20260806 21:11:28.770923 17478 heartbeater.cc:347] Connected to a master server at 127.16.178.62:36645
I20260806 21:11:28.771174 17478 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.16.178.2@KRBTEST.COM; defaulting to no policy
I20260806 21:11:28.771683 17478 heartbeater.cc:511] Master 127.16.178.62:36645 requested a full tablet report, sending...
I20260806 21:11:28.773838 17176 ts_manager.cc:194] Registered new tserver with Master: 332863ed73d049bbbe90e1590f82fa2c (127.16.178.1:42611)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.2@KRBTEST.COM" created.
I20260806 21:11:28.775446 17176 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.1@KRBTEST.COM'} at 127.16.178.1:52041
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab.
Entry for principal kudu/127.16.178.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab.
I20260806 21:11:28.791524 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.2:0
--local_ip_for_outbound_sockets=127.16.178.2
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:36645
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab
--principal=kudu/127.16.178.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:46781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:11:28.910718 17486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:28.911023 17486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:28.911077 17486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:28.915277 17486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:28.915359 17486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:28.915428 17486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.2
I20260806 21:11:28.921115 17486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:46781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab
--principal=kudu/127.16.178.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:36645
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.2
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:28.922272 17486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:28.923323 17486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:28.933503 17493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:28.933668 17486 server_base.cc:1034] running on GCE node
W20260806 21:11:28.933552 17496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:28.933542 17492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:28.934228 17486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:28.934950 17486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:28.936136 17486 hybrid_clock.cc:648] HybridClock initialized: now 1786050688936114 us; error 35 us; skew 500 ppm
Aug 06 21:11:28 dist-test-slave-lgr5 krb5kdc[17103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050688, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:28.938931 17486 init.cc:377] Logged in from keytab as kudu/127.16.178.2@KRBTEST.COM (short username kudu)
I20260806 21:11:28.939852 17486 webserver.cc:460] Webserver started at http://127.16.178.2:38269/ using document root <none> and password file <none>
I20260806 21:11:28.940455 17486 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:28.940551 17486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:28.940759 17486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:28.942801 17486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-1/data/instance:
uuid: "a9b1c6dae92f4837b43c5896654b37e3"
format_stamp: "Formatted at 2026-08-06 21:11:28 on dist-test-slave-lgr5"
server_key: "0ec16f0ff566dc09bd355808885f8479"
server_key_iv: "3a19a062c82bff87a4629783c9e71712"
server_key_version: "encryptionkey@0"
I20260806 21:11:28.943462 17486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance:
uuid: "a9b1c6dae92f4837b43c5896654b37e3"
format_stamp: "Formatted at 2026-08-06 21:11:28 on dist-test-slave-lgr5"
server_key: "0ec16f0ff566dc09bd355808885f8479"
server_key_iv: "3a19a062c82bff87a4629783c9e71712"
server_key_version: "encryptionkey@0"
I20260806 21:11:28.947544 17486 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 21:11:28.950026 17505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:28.951243 17486 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 21:11:28.951372 17486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-1/wal
uuid: "a9b1c6dae92f4837b43c5896654b37e3"
format_stamp: "Formatted at 2026-08-06 21:11:28 on dist-test-slave-lgr5"
server_key: "0ec16f0ff566dc09bd355808885f8479"
server_key_iv: "3a19a062c82bff87a4629783c9e71712"
server_key_version: "encryptionkey@0"
I20260806 21:11:28.951478 17486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:28.961374 17486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:28.964849 17486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:28.965049 17486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:28.965682 17486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:11:28.966606 17486 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:11:28.966667 17486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:28.966730 17486 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:11:28.966758 17486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:28.980067 17486 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.2:35609
I20260806 21:11:28.980108 17677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.2:35609 every 8 connection(s)
I20260806 21:11:28.981174 17486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
I20260806 21:11:28.988996 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 17486
I20260806 21:11:28.989108 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance
I20260806 21:11:28.989351 17096 external_mini_cluster.cc:1428] Setting key 24eb4525df4cf623971f7222a275ae53
Aug 06 21:11:28 dist-test-slave-lgr5 krb5kdc[17103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050688, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.2@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:11:28.996599 17233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:28.983513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:47863 (local address 127.16.178.62:36645)
0806 21:11:28.983694 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:28.983699 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:28.984751 (+ 1052us) server_negotiation.cc:407] Connection header received
0806 21:11:28.986034 (+ 1283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:28.986037 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:28.986105 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:28.986181 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:28.987525 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:28.988171 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:28.989427 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:28.989712 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:28.989823 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:28.992759 (+ 2936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:28.992763 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:28.992766 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:28.992801 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:28.994340 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:28.994862 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:28.994866 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:28.994868 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:28.994970 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:28.995370 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:28.995372 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:28.995373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:28.995565 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:28.995637 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:28.996339 (+ 702us) server_negotiation.cc:299] Negotiation successful
0806 21:11:28.996469 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 21:11:28.997871 17680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:28.983950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:36645 (local address 127.16.178.2:47863)
0806 21:11:28.984597 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:28.984642 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:28.985815 (+ 1173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:28.986336 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:28.986342 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:28.986820 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:28.987370 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:28.987386 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:28.988309 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:28.988312 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:28.989284 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:28.989295 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:28.989881 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:28.989885 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:28.990079 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:28.990834 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:28.990855 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:28.992571 (+ 1716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:28.994480 (+ 1909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:28.994485 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:28.994499 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:28.994763 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:28.995075 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:28.995078 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:28.995080 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:28.995275 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:28.995658 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:28.995666 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:28.996033 (+ 367us) client_negotiation.cc:769] Sending connection context
0806 21:11:28.996312 (+ 279us) client_negotiation.cc:240] Negotiation successful
0806 21:11:28.996693 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":289,"threads_started":1}
I20260806 21:11:28.999313 17678 heartbeater.cc:347] Connected to a master server at 127.16.178.62:36645
I20260806 21:11:28.999593 17678 heartbeater.cc:464] Registering TS with master...
I20260806 21:11:29.000201 17678 heartbeater.cc:511] Master 127.16.178.62:36645 requested a full tablet report, sending...
I20260806 21:11:29.001519 17176 ts_manager.cc:194] Registered new tserver with Master: a9b1c6dae92f4837b43c5896654b37e3 (127.16.178.2:35609)
I20260806 21:11:29.002337 17176 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.2@KRBTEST.COM'} at 127.16.178.2:47863
WARNING: no policy specified for kudu/127.16.178.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab.
Entry for principal kudu/127.16.178.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab.
I20260806 21:11:29.022818 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.3:0
--local_ip_for_outbound_sockets=127.16.178.3
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:36645
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab
--principal=kudu/127.16.178.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:46781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:11:29.146142 17684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:29.146487 17684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:29.146534 17684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:29.150774 17684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:29.150861 17684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:29.150925 17684 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.3
I20260806 21:11:29.156766 17684 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:46781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab
--principal=kudu/127.16.178.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:36645
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17684
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.3
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:29.158195 17684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:29.159371 17684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:29.170007 17693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:29.169998 17695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:29.170158 17684 server_base.cc:1034] running on GCE node
W20260806 21:11:29.169998 17692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:29.170637 17684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:29.171344 17684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:29.172519 17684 hybrid_clock.cc:648] HybridClock initialized: now 1786050689172497 us; error 35 us; skew 500 ppm
Aug 06 21:11:29 dist-test-slave-lgr5 krb5kdc[17103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050689, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:29.175710 17684 init.cc:377] Logged in from keytab as kudu/127.16.178.3@KRBTEST.COM (short username kudu)
I20260806 21:11:29.176951 17684 webserver.cc:460] Webserver started at http://127.16.178.3:36215/ using document root <none> and password file <none>
I20260806 21:11:29.177605 17684 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:29.177680 17684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:29.177901 17684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:29.180325 17684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-2/data/instance:
uuid: "375c2784d40d43199fb34e9d814f89b7"
format_stamp: "Formatted at 2026-08-06 21:11:29 on dist-test-slave-lgr5"
server_key: "c72b9008883c8e404e5f94213cd490d5"
server_key_iv: "9d8c2b705f841b2461a53371b3974da9"
server_key_version: "encryptionkey@0"
I20260806 21:11:29.180943 17684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance:
uuid: "375c2784d40d43199fb34e9d814f89b7"
format_stamp: "Formatted at 2026-08-06 21:11:29 on dist-test-slave-lgr5"
server_key: "c72b9008883c8e404e5f94213cd490d5"
server_key_iv: "9d8c2b705f841b2461a53371b3974da9"
server_key_version: "encryptionkey@0"
I20260806 21:11:29.185513 17684 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:11:29.188436 17708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:29.189906 17684 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 21:11:29.190048 17684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-2/wal
uuid: "375c2784d40d43199fb34e9d814f89b7"
format_stamp: "Formatted at 2026-08-06 21:11:29 on dist-test-slave-lgr5"
server_key: "c72b9008883c8e404e5f94213cd490d5"
server_key_iv: "9d8c2b705f841b2461a53371b3974da9"
server_key_version: "encryptionkey@0"
I20260806 21:11:29.190172 17684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:29.210425 17684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:29.213701 17684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:29.213896 17684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:29.214517 17684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:11:29.215404 17684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:11:29.215461 17684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:29.215515 17684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:11:29.215541 17684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:29.228731 17684 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.3:44505
I20260806 21:11:29.228760 17883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.3:44505 every 8 connection(s)
I20260806 21:11:29.229787 17684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
I20260806 21:11:29.231328 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 17684
I20260806 21:11:29.231451 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance
I20260806 21:11:29.231706 17096 external_mini_cluster.cc:1428] Setting key ed01ba22a216a46a6475be0b16febaff
Aug 06 21:11:29 dist-test-slave-lgr5 krb5kdc[17103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050689, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.3@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:11:29.243826 17233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:29.231987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:50235 (local address 127.16.178.62:36645)
0806 21:11:29.232176 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:29.232181 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:29.233161 (+ 980us) server_negotiation.cc:407] Connection header received
0806 21:11:29.234538 (+ 1377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:29.234540 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:29.234589 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:29.234654 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:29.235836 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:29.236438 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:29.237587 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:29.237808 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:29.237886 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:29.240406 (+ 2520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:29.240409 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:29.240411 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:29.240437 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:29.241754 (+ 1317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:29.242266 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:29.242269 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:29.242270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:29.242357 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:29.242688 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:29.242691 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:29.242693 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:29.242898 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:29.242984 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:29.243518 (+ 534us) server_negotiation.cc:299] Negotiation successful
0806 21:11:29.243654 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 21:11:29.244735 17888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:29.232506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:36645 (local address 127.16.178.3:50235)
0806 21:11:29.233025 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:29.233070 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:29.234330 (+ 1260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:29.234809 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:29.234815 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:29.235195 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:29.235701 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:29.235715 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:29.236570 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:29.236572 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:29.237465 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:29.237475 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:29.237923 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:29.237925 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:29.238074 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:29.238613 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:29.238629 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:29.240213 (+ 1584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:29.241875 (+ 1662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:29.241882 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:29.241900 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:29.242167 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:29.242442 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:29.242445 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:29.242447 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:29.242598 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:29.243000 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:29.243007 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:29.243269 (+ 262us) client_negotiation.cc:769] Sending connection context
0806 21:11:29.243459 (+ 190us) client_negotiation.cc:240] Negotiation successful
0806 21:11:29.243816 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":124,"threads_started":1}
I20260806 21:11:29.246057 17884 heartbeater.cc:347] Connected to a master server at 127.16.178.62:36645
I20260806 21:11:29.246332 17884 heartbeater.cc:464] Registering TS with master...
I20260806 21:11:29.246912 17884 heartbeater.cc:511] Master 127.16.178.62:36645 requested a full tablet report, sending...
I20260806 21:11:29.248025 17175 ts_manager.cc:194] Registered new tserver with Master: 375c2784d40d43199fb34e9d814f89b7 (127.16.178.3:44505)
I20260806 21:11:29.248648 17175 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.3@KRBTEST.COM'} at 127.16.178.3:50235
I20260806 21:11:29.258599 17096 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 21:11:29 dist-test-slave-lgr5 krb5kdc[17103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050689, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 21:11:29 dist-test-slave-lgr5 krb5kdc[17103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050689, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.178.1@KRBTEST.COM
I20260806 21:11:29.289263 17898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:29.278758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56432 (local address 127.16.178.1:42611)
0806 21:11:29.279116 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:29.279123 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:29.279151 (+ 28us) server_negotiation.cc:407] Connection header received
0806 21:11:29.279265 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:29.279274 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:29.279510 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:29.279638 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:29.280463 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:29.281277 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:29.282262 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:29.282727 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:29.282822 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:29.285378 (+ 2556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:29.285392 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:29.285408 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:29.285437 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:29.287130 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:29.287599 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:29.287606 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:29.287607 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:29.287719 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:29.288135 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:29.288139 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:29.288141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:29.288463 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:29.288639 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:29.288917 (+ 278us) server_negotiation.cc:299] Negotiation successful
0806 21:11:29.289090 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":119,"threads_started":1}
W20260806 21:11:29.290431 17336 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56432
Aug 06 21:11:29 dist-test-slave-lgr5 krb5kdc[17103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 21:11:29 dist-test-slave-lgr5 krb5kdc[17103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.178.1@KRBTEST.COM
I20260806 21:11:29.321286 17898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:29.313105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56444 (local address 127.16.178.1:42611)
0806 21:11:29.313290 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:29.313294 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:29.313364 (+ 70us) server_negotiation.cc:407] Connection header received
0806 21:11:29.313503 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:29.313506 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:29.313553 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:29.313626 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:29.314312 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:29.314823 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:29.315620 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:29.315841 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:29.315920 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:29.318408 (+ 2488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:29.318411 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:29.318414 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:29.318439 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:29.319794 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:29.320314 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:29.320317 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:29.320318 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:29.320399 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:29.320589 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:29.320591 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:29.320592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:29.320759 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:29.320863 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:29.321021 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 21:11:29.321129 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 21:11:29.322654 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 17275
I20260806 21:11:29.337162 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 17486
I20260806 21:11:29.350906 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 17684
I20260806 21:11:29.360472 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 17120
2026-08-06T21:11:29Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2524 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:11:29 dist-test-slave-lgr5 krb5kdc[17912](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:11:29 dist-test-slave-lgr5 krb5kdc[17912](info): set up 2 sockets
Aug 06 21:11:29 dist-test-slave-lgr5 krb5kdc[17912](info): commencing operation
krb5kdc: starting...
W20260806 21:11:30.516289 17096 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Aug 06 21:11:30 dist-test-slave-lgr5 krb5kdc[17912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:11:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:11:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.178.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
I20260806 21:11:30.642967 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:42969
--webserver_interface=127.16.178.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:38275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:42969
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:11:30.756997 17930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:30.757351 17930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:30.757423 17930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:30.761595 17930 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:11:30.761682 17930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:30.761709 17930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:30.761730 17930 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:11:30.761750 17930 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:11:30.768270 17930 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:38275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:42969
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:42969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.178.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17930
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:30.769457 17930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:30.770540 17930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:30.779479 17945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:30.779711 17930 server_base.cc:1034] running on GCE node
W20260806 21:11:30.779470 17942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:30.779477 17940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:30.780632 17930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:30.781982 17930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:30.783216 17930 hybrid_clock.cc:648] HybridClock initialized: now 1786050690783138 us; error 100 us; skew 500 ppm
Aug 06 21:11:30 dist-test-slave-lgr5 krb5kdc[17912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050690, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:30.787101 17930 init.cc:377] Logged in from keytab as kudu/127.16.178.62@KRBTEST.COM (short username kudu)
I20260806 21:11:30.788489 17930 webserver.cc:460] Webserver started at http://127.16.178.62:33991/ using document root <none> and password file <none>
I20260806 21:11:30.789227 17930 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:30.789306 17930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:30.789547 17930 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:30.791872 17930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "e5ffcd6379be41ee8d5671971570640d"
format_stamp: "Formatted at 2026-08-06 21:11:30 on dist-test-slave-lgr5"
server_key: "90df48185193bfcfe64dc0e6eac27754"
server_key_iv: "a22028bcd1caa3b09fb4e3ec3eae800a"
server_key_version: "encryptionkey@0"
I20260806 21:11:30.792567 17930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "e5ffcd6379be41ee8d5671971570640d"
format_stamp: "Formatted at 2026-08-06 21:11:30 on dist-test-slave-lgr5"
server_key: "90df48185193bfcfe64dc0e6eac27754"
server_key_iv: "a22028bcd1caa3b09fb4e3ec3eae800a"
server_key_version: "encryptionkey@0"
I20260806 21:11:30.796794 17930 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:11:30.799556 17962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:30.800956 17930 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:11:30.801092 17930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "e5ffcd6379be41ee8d5671971570640d"
format_stamp: "Formatted at 2026-08-06 21:11:30 on dist-test-slave-lgr5"
server_key: "90df48185193bfcfe64dc0e6eac27754"
server_key_iv: "a22028bcd1caa3b09fb4e3ec3eae800a"
server_key_version: "encryptionkey@0"
I20260806 21:11:30.801214 17930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:30.817586 17930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:30.820873 17930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:30.821070 17930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:30.827881 17930 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.62:42969
I20260806 21:11:30.827926 18043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.62:42969 every 8 connection(s)
I20260806 21:11:30.829185 17930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:11:30.829826 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 17930
I20260806 21:11:30.829936 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:11:30.830206 17096 external_mini_cluster.cc:1428] Setting key baf562327bb995e5cc67eaccc0e85d7e
I20260806 21:11:30.832468 18044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:30.839133 18044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5ffcd6379be41ee8d5671971570640d: Bootstrap starting.
Aug 06 21:11:30 dist-test-slave-lgr5 krb5kdc[17912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:11:30.841692 18044 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e5ffcd6379be41ee8d5671971570640d: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:30.842442 18044 log.cc:824] T 00000000000000000000000000000000 P e5ffcd6379be41ee8d5671971570640d: Log is configured to *not* fsync() on all Append() calls
I20260806 21:11:30.844574 18048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:30.831778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35338 (local address 127.16.178.62:42969)
0806 21:11:30.832227 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:30.832245 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:30.832283 (+ 38us) server_negotiation.cc:407] Connection header received
0806 21:11:30.832982 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:30.833000 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:30.833349 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:30.833638 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:30.834548 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:30.835557 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:30.836704 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:30.837173 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:30.837298 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:30.840024 (+ 2726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:30.840031 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:30.840044 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:30.840075 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:30.841686 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:30.842164 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:30.842170 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:30.842175 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:30.842291 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:30.842553 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:30.842556 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:30.842558 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:30.842976 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:30.843173 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:30.843472 (+ 299us) server_negotiation.cc:299] Negotiation successful
0806 21:11:30.843778 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":122,"threads_started":1}
I20260806 21:11:30.844597 18044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5ffcd6379be41ee8d5671971570640d: No bootstrap required, opened a new log
I20260806 21:11:30.848868 18044 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5ffcd6379be41ee8d5671971570640d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5ffcd6379be41ee8d5671971570640d" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 42969 } }
I20260806 21:11:30.849143 18044 raft_consensus.cc:374] T 00000000000000000000000000000000 P e5ffcd6379be41ee8d5671971570640d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:30.849217 18044 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5ffcd6379be41ee8d5671971570640d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5ffcd6379be41ee8d5671971570640d, State: Initialized, Role: FOLLOWER
I20260806 21:11:30.849738 18044 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5ffcd6379be41ee8d5671971570640d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5ffcd6379be41ee8d5671971570640d" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 42969 } }
I20260806 21:11:30.849920 18044 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5ffcd6379be41ee8d5671971570640d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:11:30.849998 18044 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5ffcd6379be41ee8d5671971570640d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:11:30.850159 18044 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5ffcd6379be41ee8d5671971570640d [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:30.851202 18044 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5ffcd6379be41ee8d5671971570640d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5ffcd6379be41ee8d5671971570640d" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 42969 } }
I20260806 21:11:30.851563 18044 leader_election.cc:302] T 00000000000000000000000000000000 P e5ffcd6379be41ee8d5671971570640d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5ffcd6379be41ee8d5671971570640d; no voters:
I20260806 21:11:30.851876 18044 leader_election.cc:288] T 00000000000000000000000000000000 P e5ffcd6379be41ee8d5671971570640d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:11:30.852030 18051 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5ffcd6379be41ee8d5671971570640d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:11:30.852398 18051 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5ffcd6379be41ee8d5671971570640d [term 1 LEADER]: Becoming Leader. State: Replica: e5ffcd6379be41ee8d5671971570640d, State: Running, Role: LEADER
I20260806 21:11:30.852768 18051 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5ffcd6379be41ee8d5671971570640d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5ffcd6379be41ee8d5671971570640d" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 42969 } }
I20260806 21:11:30.853279 18044 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5ffcd6379be41ee8d5671971570640d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:11:30.854615 18053 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5ffcd6379be41ee8d5671971570640d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5ffcd6379be41ee8d5671971570640d. Latest consensus state: current_term: 1 leader_uuid: "e5ffcd6379be41ee8d5671971570640d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5ffcd6379be41ee8d5671971570640d" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 42969 } } }
I20260806 21:11:30.854765 18053 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5ffcd6379be41ee8d5671971570640d [sys.catalog]: This master's current role is: LEADER
I20260806 21:11:30.855147 18052 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5ffcd6379be41ee8d5671971570640d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e5ffcd6379be41ee8d5671971570640d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5ffcd6379be41ee8d5671971570640d" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 42969 } } }
I20260806 21:11:30.855248 18052 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5ffcd6379be41ee8d5671971570640d [sys.catalog]: This master's current role is: LEADER
W20260806 21:11:30.858259 18074 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e5ffcd6379be41ee8d5671971570640d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 21:11:30.858413 18074 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 21:11:30.858567 18067 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:11:30.859359 18067 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:11:30.866590 18067 catalog_manager.cc:1347] Generated new cluster ID: 24a5e92fdfa94f3d8bd514aefaeda0a2
I20260806 21:11:30.866705 18067 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:11:30.879793 18067 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:11:30.881026 18067 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:11:30.890998 18067 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e5ffcd6379be41ee8d5671971570640d: Generated new TSK 0
I20260806 21:11:30.891860 18067 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.178.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab.
Entry for principal kudu/127.16.178.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab.
I20260806 21:11:30.929282 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.1:0
--local_ip_for_outbound_sockets=127.16.178.1
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:42969
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab
--principal=kudu/127.16.178.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:38275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:11:31.062728 18086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:31.063092 18086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:31.063141 18086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:31.067435 18086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:31.067525 18086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:31.067596 18086 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.1
I20260806 21:11:31.073438 18086 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:38275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab
--principal=kudu/127.16.178.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:42969
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18086
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.1
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:31.074653 18086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:31.075769 18086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:31.085973 18099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:31.085940 18102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:31.085940 18104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:31.086053 18086 server_base.cc:1034] running on GCE node
I20260806 21:11:31.086715 18086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:31.087519 18086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:31.088686 18086 hybrid_clock.cc:648] HybridClock initialized: now 1786050691088667 us; error 37 us; skew 500 ppm
Aug 06 21:11:31 dist-test-slave-lgr5 krb5kdc[17912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050691, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:31.091624 18086 init.cc:377] Logged in from keytab as kudu/127.16.178.1@KRBTEST.COM (short username kudu)
I20260806 21:11:31.092621 18086 webserver.cc:460] Webserver started at http://127.16.178.1:38837/ using document root <none> and password file <none>
I20260806 21:11:31.093235 18086 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:31.093312 18086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:31.093550 18086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:31.096024 18086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-0/data/instance:
uuid: "e5cec2aa85594c8c87042c8553263b99"
format_stamp: "Formatted at 2026-08-06 21:11:31 on dist-test-slave-lgr5"
server_key: "27c413e523a144c5ceee0555c720515d"
server_key_iv: "cf8bd803e0e841a449b3a1db1fab28c7"
server_key_version: "encryptionkey@0"
I20260806 21:11:31.096742 18086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance:
uuid: "e5cec2aa85594c8c87042c8553263b99"
format_stamp: "Formatted at 2026-08-06 21:11:31 on dist-test-slave-lgr5"
server_key: "27c413e523a144c5ceee0555c720515d"
server_key_iv: "cf8bd803e0e841a449b3a1db1fab28c7"
server_key_version: "encryptionkey@0"
I20260806 21:11:31.101413 18086 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:11:31.104487 18112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:31.105903 18086 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:11:31.106050 18086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-0/wal
uuid: "e5cec2aa85594c8c87042c8553263b99"
format_stamp: "Formatted at 2026-08-06 21:11:31 on dist-test-slave-lgr5"
server_key: "27c413e523a144c5ceee0555c720515d"
server_key_iv: "cf8bd803e0e841a449b3a1db1fab28c7"
server_key_version: "encryptionkey@0"
I20260806 21:11:31.106171 18086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:31.136075 18086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:31.140115 18086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:31.140334 18086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:31.141022 18086 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:11:31.141981 18086 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:11:31.142033 18086 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:31.142092 18086 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:11:31.142120 18086 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:31.152333 18086 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.1:40247
I20260806 21:11:31.152364 18275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.1:40247 every 8 connection(s)
I20260806 21:11:31.153494 18086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
I20260806 21:11:31.158447 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 18086
I20260806 21:11:31.158555 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance
I20260806 21:11:31.158794 17096 external_mini_cluster.cc:1428] Setting key 0dee39cf098b6eefe4c42f7fed0a7b77
Aug 06 21:11:31 dist-test-slave-lgr5 krb5kdc[17912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050691, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:11:31.168388 18048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:31.155570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:59877 (local address 127.16.178.62:42969)
0806 21:11:31.155701 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:31.155705 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:31.156653 (+ 948us) server_negotiation.cc:407] Connection header received
0806 21:11:31.158007 (+ 1354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:31.158011 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:31.158083 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:31.158158 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:31.159512 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:31.160114 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:31.161675 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:31.161916 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:31.161998 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:31.164582 (+ 2584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:31.164586 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:31.164589 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:31.164619 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:31.166272 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:31.166781 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:31.166784 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:31.166786 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:31.166879 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:31.167189 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:31.167192 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:31.167194 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:31.167454 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:31.167557 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:31.168036 (+ 479us) server_negotiation.cc:299] Negotiation successful
0806 21:11:31.168191 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
WARNING: no policy specified for kudu/127.16.178.2@KRBTEST.COM; defaulting to no policy
I20260806 21:11:31.169239 18280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:31.155981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:42969 (local address 127.16.178.1:59877)
0806 21:11:31.156484 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:31.156530 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:31.157776 (+ 1246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:31.158342 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:31.158349 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:31.158769 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:31.159369 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:31.159386 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:31.160246 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:31.160250 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:31.161539 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:31.161552 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:31.162071 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:31.162075 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:31.162276 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:31.162902 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:31.162919 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:31.164396 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:31.166402 (+ 2006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:31.166408 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:31.166422 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:31.166686 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:31.166975 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:31.166977 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:31.166978 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:31.167109 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:31.167544 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:31.167549 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:31.167798 (+ 249us) client_negotiation.cc:769] Sending connection context
0806 21:11:31.167956 (+ 158us) client_negotiation.cc:240] Negotiation successful
0806 21:11:31.168296 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":122,"threads_started":1}
I20260806 21:11:31.170534 18276 heartbeater.cc:347] Connected to a master server at 127.16.178.62:42969
I20260806 21:11:31.170791 18276 heartbeater.cc:464] Registering TS with master...
I20260806 21:11:31.171401 18276 heartbeater.cc:511] Master 127.16.178.62:42969 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.2@KRBTEST.COM" created.
I20260806 21:11:31.172992 17983 ts_manager.cc:194] Registered new tserver with Master: e5cec2aa85594c8c87042c8553263b99 (127.16.178.1:40247)
I20260806 21:11:31.174132 17983 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.1@KRBTEST.COM'} at 127.16.178.1:59877
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab.
Entry for principal kudu/127.16.178.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab.
I20260806 21:11:31.186985 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.2:0
--local_ip_for_outbound_sockets=127.16.178.2
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:42969
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab
--principal=kudu/127.16.178.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:38275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:11:31.314067 18285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:31.314365 18285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:31.314414 18285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:31.318634 18285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:31.318722 18285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:31.318789 18285 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.2
I20260806 21:11:31.324630 18285 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:38275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab
--principal=kudu/127.16.178.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:42969
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.2
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:31.325860 18285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:31.326969 18285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:31.338342 18292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:31.338356 18293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:31.338353 18296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:31.338474 18285 server_base.cc:1034] running on GCE node
I20260806 21:11:31.339143 18285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:31.339902 18285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:31.341068 18285 hybrid_clock.cc:648] HybridClock initialized: now 1786050691341047 us; error 71 us; skew 500 ppm
Aug 06 21:11:31 dist-test-slave-lgr5 krb5kdc[17912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050691, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:31.344331 18285 init.cc:377] Logged in from keytab as kudu/127.16.178.2@KRBTEST.COM (short username kudu)
I20260806 21:11:31.345395 18285 webserver.cc:460] Webserver started at http://127.16.178.2:43243/ using document root <none> and password file <none>
I20260806 21:11:31.345988 18285 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:31.346063 18285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:31.346249 18285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:31.348529 18285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-1/data/instance:
uuid: "3633ed033bb3436db59b6688c05278b4"
format_stamp: "Formatted at 2026-08-06 21:11:31 on dist-test-slave-lgr5"
server_key: "3589687ca1993e0254961a1b5f6c24d5"
server_key_iv: "7f19caade7de7d07a5fdcf3fc6966ee9"
server_key_version: "encryptionkey@0"
I20260806 21:11:31.349212 18285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance:
uuid: "3633ed033bb3436db59b6688c05278b4"
format_stamp: "Formatted at 2026-08-06 21:11:31 on dist-test-slave-lgr5"
server_key: "3589687ca1993e0254961a1b5f6c24d5"
server_key_iv: "7f19caade7de7d07a5fdcf3fc6966ee9"
server_key_version: "encryptionkey@0"
I20260806 21:11:31.353729 18285 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:11:31.356694 18308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:31.357978 18285 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 21:11:31.358095 18285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-1/wal
uuid: "3633ed033bb3436db59b6688c05278b4"
format_stamp: "Formatted at 2026-08-06 21:11:31 on dist-test-slave-lgr5"
server_key: "3589687ca1993e0254961a1b5f6c24d5"
server_key_iv: "7f19caade7de7d07a5fdcf3fc6966ee9"
server_key_version: "encryptionkey@0"
I20260806 21:11:31.358201 18285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:31.402218 18285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:31.406349 18285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:31.406546 18285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:31.407212 18285 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:11:31.408170 18285 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:11:31.408233 18285 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:31.408278 18285 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:11:31.408293 18285 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:31.418701 18285 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.2:42187
I20260806 21:11:31.418716 18463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.2:42187 every 8 connection(s)
I20260806 21:11:31.419776 18285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
I20260806 21:11:31.425832 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 18285
I20260806 21:11:31.425997 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance
I20260806 21:11:31.426359 17096 external_mini_cluster.cc:1428] Setting key 1fa342568bb314287ebc303175460eff
Aug 06 21:11:31 dist-test-slave-lgr5 krb5kdc[17912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050691, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.2@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:11:31.440647 18048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:31.422118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:57165 (local address 127.16.178.62:42969)
0806 21:11:31.422304 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:31.422307 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:31.423617 (+ 1310us) server_negotiation.cc:407] Connection header received
0806 21:11:31.425544 (+ 1927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:31.425549 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:31.425637 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:31.425750 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:31.427891 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:31.428715 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:31.430479 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:31.430876 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:31.431042 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:31.435209 (+ 4167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:31.435213 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:31.435216 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:31.435268 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:31.437127 (+ 1859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:31.438093 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:31.438098 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:31.438099 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:31.438226 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:31.438792 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:31.438796 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:31.438797 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:31.439105 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:31.439206 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:31.440297 (+ 1091us) server_negotiation.cc:299] Negotiation successful
0806 21:11:31.440468 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
WARNING: no policy specified for kudu/127.16.178.3@KRBTEST.COM; defaulting to no policy
I20260806 21:11:31.442024 18466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:31.422663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:42969 (local address 127.16.178.2:57165)
0806 21:11:31.423326 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:31.423420 (+ 94us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:31.425168 (+ 1748us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:31.426144 (+ 976us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:31.426161 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:31.426803 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:31.427661 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:31.427683 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:31.428933 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:31.428938 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:31.430204 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:31.430224 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:31.431172 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:31.431179 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:31.431517 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:31.432524 (+ 1007us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:31.432552 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:31.434852 (+ 2300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:31.437332 (+ 2480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:31.437344 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:31.437375 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:31.437913 (+ 538us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:31.438349 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:31.438353 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:31.438355 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:31.438638 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:31.439244 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:31.439254 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:31.439827 (+ 573us) client_negotiation.cc:769] Sending connection context
0806 21:11:31.440245 (+ 418us) client_negotiation.cc:240] Negotiation successful
0806 21:11:31.440824 (+ 579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":166,"threads_started":1}
I20260806 21:11:31.443889 18464 heartbeater.cc:347] Connected to a master server at 127.16.178.62:42969
I20260806 21:11:31.444278 18464 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.3@KRBTEST.COM" created.
I20260806 21:11:31.444926 18464 heartbeater.cc:511] Master 127.16.178.62:42969 requested a full tablet report, sending...
I20260806 21:11:31.446494 17983 ts_manager.cc:194] Registered new tserver with Master: 3633ed033bb3436db59b6688c05278b4 (127.16.178.2:42187)
I20260806 21:11:31.447438 17983 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.2@KRBTEST.COM'} at 127.16.178.2:57165
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab.
Entry for principal kudu/127.16.178.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab.
I20260806 21:11:31.464376 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.3:0
--local_ip_for_outbound_sockets=127.16.178.3
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:42969
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab
--principal=kudu/127.16.178.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:38275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:11:31.585350 18469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:31.585652 18469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:31.585698 18469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:31.589869 18469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:31.589946 18469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:31.590006 18469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.3
I20260806 21:11:31.595569 18469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:38275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab
--principal=kudu/127.16.178.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:42969
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.3
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:31.596830 18469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:31.597887 18469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:31.608115 18478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:31.608115 18475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:31.608132 18476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:31.608299 18469 server_base.cc:1034] running on GCE node
I20260806 21:11:31.608935 18469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:31.609710 18469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:31.610878 18469 hybrid_clock.cc:648] HybridClock initialized: now 1786050691610858 us; error 35 us; skew 500 ppm
Aug 06 21:11:31 dist-test-slave-lgr5 krb5kdc[17912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050691, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:31.613773 18469 init.cc:377] Logged in from keytab as kudu/127.16.178.3@KRBTEST.COM (short username kudu)
I20260806 21:11:31.614691 18469 webserver.cc:460] Webserver started at http://127.16.178.3:36629/ using document root <none> and password file <none>
I20260806 21:11:31.615271 18469 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:31.615334 18469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:31.615516 18469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:31.617789 18469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-2/data/instance:
uuid: "fe6fcb4113594c98bcd2b9afa3804d68"
format_stamp: "Formatted at 2026-08-06 21:11:31 on dist-test-slave-lgr5"
server_key: "988666c809c7dbe086a4dfb7df6a3d8f"
server_key_iv: "7821234c3fe085cf60196e597f9cf65b"
server_key_version: "encryptionkey@0"
I20260806 21:11:31.618386 18469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance:
uuid: "fe6fcb4113594c98bcd2b9afa3804d68"
format_stamp: "Formatted at 2026-08-06 21:11:31 on dist-test-slave-lgr5"
server_key: "988666c809c7dbe086a4dfb7df6a3d8f"
server_key_iv: "7821234c3fe085cf60196e597f9cf65b"
server_key_version: "encryptionkey@0"
I20260806 21:11:31.622485 18469 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:11:31.625363 18491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:31.626696 18469 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:11:31.626824 18469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-2/wal
uuid: "fe6fcb4113594c98bcd2b9afa3804d68"
format_stamp: "Formatted at 2026-08-06 21:11:31 on dist-test-slave-lgr5"
server_key: "988666c809c7dbe086a4dfb7df6a3d8f"
server_key_iv: "7821234c3fe085cf60196e597f9cf65b"
server_key_version: "encryptionkey@0"
I20260806 21:11:31.626956 18469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:31.643751 18469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:31.646983 18469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:31.647182 18469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:31.647799 18469 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:11:31.648733 18469 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:11:31.648783 18469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:31.648829 18469 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:11:31.648859 18469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:31.657920 18469 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.3:40403
I20260806 21:11:31.657945 18655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.3:40403 every 8 connection(s)
I20260806 21:11:31.658874 18469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
I20260806 21:11:31.662935 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 18469
I20260806 21:11:31.663077 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance
I20260806 21:11:31.663404 17096 external_mini_cluster.cc:1428] Setting key b2ac4ce223edf1caac8ef59df54017a5
Aug 06 21:11:31 dist-test-slave-lgr5 krb5kdc[17912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050691, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.3@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:11:31.673173 18048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:31.660768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:38783 (local address 127.16.178.62:42969)
0806 21:11:31.660916 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:31.660920 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:31.661893 (+ 973us) server_negotiation.cc:407] Connection header received
0806 21:11:31.662879 (+ 986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:31.662882 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:31.662937 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:31.663011 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:31.664457 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:31.665077 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:31.666562 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:31.666796 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:31.666886 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:31.669810 (+ 2924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:31.669815 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:31.669818 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:31.669852 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:31.671072 (+ 1220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:31.671703 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:31.671706 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:31.671707 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:31.671783 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:31.672103 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:31.672107 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:31.672109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:31.672314 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:31.672394 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:31.672922 (+ 528us) server_negotiation.cc:299] Negotiation successful
0806 21:11:31.673038 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 21:11:31.674098 18658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:31.661169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:42969 (local address 127.16.178.3:38783)
0806 21:11:31.661704 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:31.661758 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:31.662569 (+ 811us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:31.663221 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:31.663232 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:31.663737 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:31.664331 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:31.664347 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:31.665219 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:31.665222 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:31.666405 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:31.666420 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:31.666966 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:31.666969 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:31.667141 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:31.667801 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:31.667822 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:31.669558 (+ 1736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:31.671292 (+ 1734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:31.671299 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:31.671315 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:31.671609 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:31.671850 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:31.671853 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:31.671854 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:31.672007 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:31.672412 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:31.672419 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:31.672708 (+ 289us) client_negotiation.cc:769] Sending connection context
0806 21:11:31.672881 (+ 173us) client_negotiation.cc:240] Negotiation successful
0806 21:11:31.673173 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":124,"threads_started":1}
I20260806 21:11:31.675407 18656 heartbeater.cc:347] Connected to a master server at 127.16.178.62:42969
I20260806 21:11:31.675643 18656 heartbeater.cc:464] Registering TS with master...
I20260806 21:11:31.676251 18656 heartbeater.cc:511] Master 127.16.178.62:42969 requested a full tablet report, sending...
I20260806 21:11:31.677409 17983 ts_manager.cc:194] Registered new tserver with Master: fe6fcb4113594c98bcd2b9afa3804d68 (127.16.178.3:40403)
I20260806 21:11:31.677891 17096 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 21:11:31.678092 17983 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.3@KRBTEST.COM'} at 127.16.178.3:38783
I20260806 21:11:31.689618 18048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:31.682546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35354 (local address 127.16.178.62:42969)
0806 21:11:31.682674 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:31.682678 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:31.682835 (+ 157us) server_negotiation.cc:407] Connection header received
0806 21:11:31.682975 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:31.682977 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:31.683024 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:31.683090 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:31.683732 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:31.684290 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:31.685079 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:31.685283 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:31.685351 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:31.686736 (+ 1385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:31.686740 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:31.686742 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:31.686765 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:31.688162 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:31.688597 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:31.688600 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:31.688601 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:31.688678 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:31.688874 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:31.688876 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:31.688877 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:31.689090 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:31.689188 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:31.689385 (+ 197us) server_negotiation.cc:299] Negotiation successful
0806 21:11:31.689481 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 21:11:31.695948 17983 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35354:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:11:31.699424 17983 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:11:31.716269 18679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:31.710814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:40247 (local address 127.0.0.1:47198)
0806 21:11:31.711498 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:31.711511 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:31.711591 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:31.712268 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:31.712271 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:31.712291 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:11:31.712389 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:31.712396 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:31.713774 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:31.713777 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:31.714925 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:31.714937 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:31.715796 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:31.715800 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:31.715897 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:31.715922 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 21:11:31.715993 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 21:11:31.716098 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"mutex_wait_us":74,"thread_start_us":105,"threads_started":1}
I20260806 21:11:31.716295 18680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:31.710950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47198 (local address 127.16.178.1:40247)
0806 21:11:31.711824 (+ 874us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:31.711831 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:31.711852 (+ 21us) server_negotiation.cc:407] Connection header received
0806 21:11:31.711935 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:31.711940 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:31.712138 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:31.712274 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:11:31.712507 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:31.713521 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:31.715052 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:31.715663 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:31.715769 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:31.715903 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:31.716027 (+ 124us) server_negotiation.cc:299] Negotiation successful
0806 21:11:31.716144 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":776,"thread_start_us":103,"threads_started":1}
I20260806 21:11:31.716401 18678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:31.710278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37488 (local address 127.16.178.3:40403)
0806 21:11:31.711290 (+ 1012us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:31.711304 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:31.711326 (+ 22us) server_negotiation.cc:407] Connection header received
0806 21:11:31.711416 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:31.711421 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:31.711620 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:31.711759 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:11:31.712118 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:31.712970 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:31.714795 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:31.715469 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:31.715554 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:31.715733 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:31.716176 (+ 443us) server_negotiation.cc:299] Negotiation successful
0806 21:11:31.716266 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":143,"threads_started":1}
I20260806 21:11:31.717159 18675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:31.710151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54848 (local address 127.16.178.2:42187)
0806 21:11:31.710774 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:31.710792 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:31.711281 (+ 489us) server_negotiation.cc:407] Connection header received
0806 21:11:31.711350 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:31.711354 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:31.711545 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:31.711684 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:11:31.711999 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:31.713195 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:31.715798 (+ 2603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:31.716634 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:31.716735 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:31.716873 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:31.716928 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 21:11:31.717014 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":117,"threads_started":1}
I20260806 21:11:31.716594 18672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:31.709998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.3:40403 (local address 127.0.0.1:37488)
0806 21:11:31.710941 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:31.710994 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:31.711126 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:31.711864 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:31.711867 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:31.711882 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:11:31.711978 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:31.711984 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:31.713098 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:31.713102 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:31.714663 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:31.714675 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:31.715602 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:31.715606 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:31.715713 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:31.715764 (+ 51us) client_negotiation.cc:769] Sending connection context
0806 21:11:31.715871 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 21:11:31.716012 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"mutex_wait_us":82,"thread_start_us":103,"threads_started":1}
I20260806 21:11:31.717039 18671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:31.709998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.2:42187 (local address 127.0.0.1:54848)
0806 21:11:31.711111 (+ 1113us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:31.711123 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:31.711195 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:31.711689 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:31.711696 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:31.711718 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:11:31.711875 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:31.711882 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:31.713335 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:31.713339 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:31.714682 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:31.714689 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:31.716746 (+ 2057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:31.716749 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:31.716826 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:31.716844 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 21:11:31.716894 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 21:11:31.716948 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1056,"mutex_wait_us":162,"thread_start_us":110,"threads_started":1}
I20260806 21:11:31.720290 18177 tablet_service.cc:1467] Processing CreateTablet for tablet ab0206b3eafe447baa3bfefa07c72b9e (DEFAULT_TABLE table=test-table [id=f8995ae9fb1a4a528489eeaaee16d3eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:11:31.721712 18177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab0206b3eafe447baa3bfefa07c72b9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:31.722167 18373 tablet_service.cc:1467] Processing CreateTablet for tablet ab0206b3eafe447baa3bfefa07c72b9e (DEFAULT_TABLE table=test-table [id=f8995ae9fb1a4a528489eeaaee16d3eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:11:31.722241 18561 tablet_service.cc:1467] Processing CreateTablet for tablet ab0206b3eafe447baa3bfefa07c72b9e (DEFAULT_TABLE table=test-table [id=f8995ae9fb1a4a528489eeaaee16d3eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:11:31.723179 18373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab0206b3eafe447baa3bfefa07c72b9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:31.723264 18561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab0206b3eafe447baa3bfefa07c72b9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:31.729202 18681 tablet_bootstrap.cc:492] T ab0206b3eafe447baa3bfefa07c72b9e P e5cec2aa85594c8c87042c8553263b99: Bootstrap starting.
I20260806 21:11:31.729573 18682 tablet_bootstrap.cc:492] T ab0206b3eafe447baa3bfefa07c72b9e P 3633ed033bb3436db59b6688c05278b4: Bootstrap starting.
I20260806 21:11:31.730613 18686 tablet_bootstrap.cc:492] T ab0206b3eafe447baa3bfefa07c72b9e P fe6fcb4113594c98bcd2b9afa3804d68: Bootstrap starting.
I20260806 21:11:31.732671 18686 tablet_bootstrap.cc:654] T ab0206b3eafe447baa3bfefa07c72b9e P fe6fcb4113594c98bcd2b9afa3804d68: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:31.732781 18681 tablet_bootstrap.cc:654] T ab0206b3eafe447baa3bfefa07c72b9e P e5cec2aa85594c8c87042c8553263b99: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:31.733464 18682 tablet_bootstrap.cc:654] T ab0206b3eafe447baa3bfefa07c72b9e P 3633ed033bb3436db59b6688c05278b4: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:31.733767 18681 log.cc:824] T ab0206b3eafe447baa3bfefa07c72b9e P e5cec2aa85594c8c87042c8553263b99: Log is configured to *not* fsync() on all Append() calls
I20260806 21:11:31.734278 18682 log.cc:824] T ab0206b3eafe447baa3bfefa07c72b9e P 3633ed033bb3436db59b6688c05278b4: Log is configured to *not* fsync() on all Append() calls
I20260806 21:11:31.735889 18681 tablet_bootstrap.cc:492] T ab0206b3eafe447baa3bfefa07c72b9e P e5cec2aa85594c8c87042c8553263b99: No bootstrap required, opened a new log
I20260806 21:11:31.736065 18686 log.cc:824] T ab0206b3eafe447baa3bfefa07c72b9e P fe6fcb4113594c98bcd2b9afa3804d68: Log is configured to *not* fsync() on all Append() calls
I20260806 21:11:31.736122 18681 ts_tablet_manager.cc:1397] T ab0206b3eafe447baa3bfefa07c72b9e P e5cec2aa85594c8c87042c8553263b99: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 21:11:31.736306 18682 tablet_bootstrap.cc:492] T ab0206b3eafe447baa3bfefa07c72b9e P 3633ed033bb3436db59b6688c05278b4: No bootstrap required, opened a new log
I20260806 21:11:31.736485 18682 ts_tablet_manager.cc:1397] T ab0206b3eafe447baa3bfefa07c72b9e P 3633ed033bb3436db59b6688c05278b4: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 21:11:31.740263 18682 raft_consensus.cc:348] T ab0206b3eafe447baa3bfefa07c72b9e P 3633ed033bb3436db59b6688c05278b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3633ed033bb3436db59b6688c05278b4" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 42187 } } peers { permanent_uuid: "fe6fcb4113594c98bcd2b9afa3804d68" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 40403 } } peers { permanent_uuid: "e5cec2aa85594c8c87042c8553263b99" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 40247 } }
I20260806 21:11:31.740567 18682 raft_consensus.cc:374] T ab0206b3eafe447baa3bfefa07c72b9e P 3633ed033bb3436db59b6688c05278b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:31.740649 18682 raft_consensus.cc:729] T ab0206b3eafe447baa3bfefa07c72b9e P 3633ed033bb3436db59b6688c05278b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3633ed033bb3436db59b6688c05278b4, State: Initialized, Role: FOLLOWER
I20260806 21:11:31.740511 18681 raft_consensus.cc:348] T ab0206b3eafe447baa3bfefa07c72b9e P e5cec2aa85594c8c87042c8553263b99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3633ed033bb3436db59b6688c05278b4" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 42187 } } peers { permanent_uuid: "fe6fcb4113594c98bcd2b9afa3804d68" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 40403 } } peers { permanent_uuid: "e5cec2aa85594c8c87042c8553263b99" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 40247 } }
I20260806 21:11:31.740738 18681 raft_consensus.cc:374] T ab0206b3eafe447baa3bfefa07c72b9e P e5cec2aa85594c8c87042c8553263b99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:31.740823 18681 raft_consensus.cc:729] T ab0206b3eafe447baa3bfefa07c72b9e P e5cec2aa85594c8c87042c8553263b99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5cec2aa85594c8c87042c8553263b99, State: Initialized, Role: FOLLOWER
I20260806 21:11:31.741154 18686 tablet_bootstrap.cc:492] T ab0206b3eafe447baa3bfefa07c72b9e P fe6fcb4113594c98bcd2b9afa3804d68: No bootstrap required, opened a new log
I20260806 21:11:31.741183 18682 consensus_queue.cc:260] T ab0206b3eafe447baa3bfefa07c72b9e P 3633ed033bb3436db59b6688c05278b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3633ed033bb3436db59b6688c05278b4" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 42187 } } peers { permanent_uuid: "fe6fcb4113594c98bcd2b9afa3804d68" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 40403 } } peers { permanent_uuid: "e5cec2aa85594c8c87042c8553263b99" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 40247 } }
I20260806 21:11:31.741227 18681 consensus_queue.cc:260] T ab0206b3eafe447baa3bfefa07c72b9e P e5cec2aa85594c8c87042c8553263b99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3633ed033bb3436db59b6688c05278b4" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 42187 } } peers { permanent_uuid: "fe6fcb4113594c98bcd2b9afa3804d68" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 40403 } } peers { permanent_uuid: "e5cec2aa85594c8c87042c8553263b99" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 40247 } }
I20260806 21:11:31.741341 18686 ts_tablet_manager.cc:1397] T ab0206b3eafe447baa3bfefa07c72b9e P fe6fcb4113594c98bcd2b9afa3804d68: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.003s
I20260806 21:11:31.741624 18691 raft_consensus.cc:482] T ab0206b3eafe447baa3bfefa07c72b9e P 3633ed033bb3436db59b6688c05278b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:11:31.741812 18691 raft_consensus.cc:504] T ab0206b3eafe447baa3bfefa07c72b9e P 3633ed033bb3436db59b6688c05278b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3633ed033bb3436db59b6688c05278b4" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 42187 } } peers { permanent_uuid: "fe6fcb4113594c98bcd2b9afa3804d68" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 40403 } } peers { permanent_uuid: "e5cec2aa85594c8c87042c8553263b99" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 40247 } }
I20260806 21:11:31.742205 18682 ts_tablet_manager.cc:1428] T ab0206b3eafe447baa3bfefa07c72b9e P 3633ed033bb3436db59b6688c05278b4: Time spent starting tablet: real 0.006s user 0.000s sys 0.005s
I20260806 21:11:31.742205 18681 ts_tablet_manager.cc:1428] T ab0206b3eafe447baa3bfefa07c72b9e P e5cec2aa85594c8c87042c8553263b99: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 21:11:31.742619 18276 heartbeater.cc:503] Master 127.16.178.62:42969 was elected leader, sending a full tablet report...
I20260806 21:11:31.745349 18464 heartbeater.cc:503] Master 127.16.178.62:42969 was elected leader, sending a full tablet report...
I20260806 21:11:31.745661 18686 raft_consensus.cc:348] T ab0206b3eafe447baa3bfefa07c72b9e P fe6fcb4113594c98bcd2b9afa3804d68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3633ed033bb3436db59b6688c05278b4" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 42187 } } peers { permanent_uuid: "fe6fcb4113594c98bcd2b9afa3804d68" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 40403 } } peers { permanent_uuid: "e5cec2aa85594c8c87042c8553263b99" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 40247 } }
I20260806 21:11:31.746182 18686 raft_consensus.cc:374] T ab0206b3eafe447baa3bfefa07c72b9e P fe6fcb4113594c98bcd2b9afa3804d68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:31.746305 18686 raft_consensus.cc:729] T ab0206b3eafe447baa3bfefa07c72b9e P fe6fcb4113594c98bcd2b9afa3804d68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe6fcb4113594c98bcd2b9afa3804d68, State: Initialized, Role: FOLLOWER
I20260806 21:11:31.746970 18686 consensus_queue.cc:260] T ab0206b3eafe447baa3bfefa07c72b9e P fe6fcb4113594c98bcd2b9afa3804d68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3633ed033bb3436db59b6688c05278b4" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 42187 } } peers { permanent_uuid: "fe6fcb4113594c98bcd2b9afa3804d68" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 40403 } } peers { permanent_uuid: "e5cec2aa85594c8c87042c8553263b99" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 40247 } }
I20260806 21:11:31.748064 18466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:31.743255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.3:40403 (local address 127.16.178.2:43207)
0806 21:11:31.743559 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:31.743575 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:31.743667 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:31.744299 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:31.744302 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:31.744321 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:11:31.744446 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:31.744453 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:31.745491 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:31.745495 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:31.746671 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:31.746683 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:31.747642 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:31.747646 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:31.747751 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:31.747774 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 21:11:31.747863 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 21:11:31.747939 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213}
I20260806 21:11:31.748147 18686 ts_tablet_manager.cc:1428] T ab0206b3eafe447baa3bfefa07c72b9e P fe6fcb4113594c98bcd2b9afa3804d68: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260806 21:11:31.748562 18656 heartbeater.cc:503] Master 127.16.178.62:42969 was elected leader, sending a full tablet report...
I20260806 21:11:31.748778 18678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:31.743822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:43207 (local address 127.16.178.3:40403)
0806 21:11:31.743946 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:31.743950 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:31.743963 (+ 13us) server_negotiation.cc:407] Connection header received
0806 21:11:31.744015 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:31.744018 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:31.744105 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:31.744174 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:11:31.744573 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:31.745335 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:31.746808 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:31.747424 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:31.747531 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:31.747583 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:31.748586 (+ 1003us) server_negotiation.cc:299] Negotiation successful
0806 21:11:31.748669 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 21:11:31.749571 18589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab0206b3eafe447baa3bfefa07c72b9e" candidate_uuid: "3633ed033bb3436db59b6688c05278b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe6fcb4113594c98bcd2b9afa3804d68" is_pre_election: true
I20260806 21:11:31.750021 18589 raft_consensus.cc:2454] T ab0206b3eafe447baa3bfefa07c72b9e P fe6fcb4113594c98bcd2b9afa3804d68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3633ed033bb3436db59b6688c05278b4 in term 0.
I20260806 21:11:31.750129 18691 leader_election.cc:288] T ab0206b3eafe447baa3bfefa07c72b9e P 3633ed033bb3436db59b6688c05278b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fe6fcb4113594c98bcd2b9afa3804d68 (127.16.178.3:40403), e5cec2aa85594c8c87042c8553263b99 (127.16.178.1:40247)
I20260806 21:11:31.750576 18313 leader_election.cc:302] T ab0206b3eafe447baa3bfefa07c72b9e P 3633ed033bb3436db59b6688c05278b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3633ed033bb3436db59b6688c05278b4, fe6fcb4113594c98bcd2b9afa3804d68; no voters:
I20260806 21:11:31.750852 18691 raft_consensus.cc:2781] T ab0206b3eafe447baa3bfefa07c72b9e P 3633ed033bb3436db59b6688c05278b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:11:31.750957 18691 raft_consensus.cc:482] T ab0206b3eafe447baa3bfefa07c72b9e P 3633ed033bb3436db59b6688c05278b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:11:31.751011 18691 raft_consensus.cc:3037] T ab0206b3eafe447baa3bfefa07c72b9e P 3633ed033bb3436db59b6688c05278b4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:31.752234 18691 raft_consensus.cc:504] T ab0206b3eafe447baa3bfefa07c72b9e P 3633ed033bb3436db59b6688c05278b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3633ed033bb3436db59b6688c05278b4" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 42187 } } peers { permanent_uuid: "fe6fcb4113594c98bcd2b9afa3804d68" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 40403 } } peers { permanent_uuid: "e5cec2aa85594c8c87042c8553263b99" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 40247 } }
I20260806 21:11:31.752565 18691 leader_election.cc:288] T ab0206b3eafe447baa3bfefa07c72b9e P 3633ed033bb3436db59b6688c05278b4 [CANDIDATE]: Term 1 election: Requested vote from peers fe6fcb4113594c98bcd2b9afa3804d68 (127.16.178.3:40403), e5cec2aa85594c8c87042c8553263b99 (127.16.178.1:40247)
I20260806 21:11:31.752825 18589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab0206b3eafe447baa3bfefa07c72b9e" candidate_uuid: "3633ed033bb3436db59b6688c05278b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe6fcb4113594c98bcd2b9afa3804d68"
I20260806 21:11:31.752947 18589 raft_consensus.cc:3037] T ab0206b3eafe447baa3bfefa07c72b9e P fe6fcb4113594c98bcd2b9afa3804d68 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:31.753449 18466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:31.749849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:40247 (local address 127.16.178.2:59703)
0806 21:11:31.749954 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:31.749968 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:31.750040 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:31.750410 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:31.750413 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:31.750430 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:11:31.750497 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:31.750503 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:31.751435 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:31.751439 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:31.752321 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:31.752335 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:31.753148 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:31.753151 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:31.753215 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:31.753234 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 21:11:31.753292 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 21:11:31.753341 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 21:11:31.753504 18680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:31.749991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:59703 (local address 127.16.178.1:40247)
0806 21:11:31.750110 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:31.750115 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:31.750129 (+ 14us) server_negotiation.cc:407] Connection header received
0806 21:11:31.750174 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:31.750177 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:31.750224 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:31.750296 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:11:31.750616 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:31.751316 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:31.752453 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:31.753037 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:31.753159 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:31.753210 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:31.753311 (+ 101us) server_negotiation.cc:299] Negotiation successful
0806 21:11:31.753388 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 21:11:31.754019 18208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab0206b3eafe447baa3bfefa07c72b9e" candidate_uuid: "3633ed033bb3436db59b6688c05278b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5cec2aa85594c8c87042c8553263b99" is_pre_election: true
I20260806 21:11:31.754037 18207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab0206b3eafe447baa3bfefa07c72b9e" candidate_uuid: "3633ed033bb3436db59b6688c05278b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5cec2aa85594c8c87042c8553263b99"
I20260806 21:11:31.754248 18589 raft_consensus.cc:2454] T ab0206b3eafe447baa3bfefa07c72b9e P fe6fcb4113594c98bcd2b9afa3804d68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3633ed033bb3436db59b6688c05278b4 in term 1.
I20260806 21:11:31.754302 18208 raft_consensus.cc:2454] T ab0206b3eafe447baa3bfefa07c72b9e P e5cec2aa85594c8c87042c8553263b99 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3633ed033bb3436db59b6688c05278b4 in term 0.
I20260806 21:11:31.754604 18313 leader_election.cc:302] T ab0206b3eafe447baa3bfefa07c72b9e P 3633ed033bb3436db59b6688c05278b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3633ed033bb3436db59b6688c05278b4, fe6fcb4113594c98bcd2b9afa3804d68; no voters:
I20260806 21:11:31.754765 18691 raft_consensus.cc:2781] T ab0206b3eafe447baa3bfefa07c72b9e P 3633ed033bb3436db59b6688c05278b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:11:31.754870 18691 raft_consensus.cc:686] T ab0206b3eafe447baa3bfefa07c72b9e P 3633ed033bb3436db59b6688c05278b4 [term 1 LEADER]: Becoming Leader. State: Replica: 3633ed033bb3436db59b6688c05278b4, State: Running, Role: LEADER
I20260806 21:11:31.755208 18691 consensus_queue.cc:237] T ab0206b3eafe447baa3bfefa07c72b9e P 3633ed033bb3436db59b6688c05278b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3633ed033bb3436db59b6688c05278b4" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 42187 } } peers { permanent_uuid: "fe6fcb4113594c98bcd2b9afa3804d68" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 40403 } } peers { permanent_uuid: "e5cec2aa85594c8c87042c8553263b99" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 40247 } }
I20260806 21:11:31.758348 17983 catalog_manager.cc:5495] T ab0206b3eafe447baa3bfefa07c72b9e P 3633ed033bb3436db59b6688c05278b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3633ed033bb3436db59b6688c05278b4 (127.16.178.2). New cstate: current_term: 1 leader_uuid: "3633ed033bb3436db59b6688c05278b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3633ed033bb3436db59b6688c05278b4" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 42187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fe6fcb4113594c98bcd2b9afa3804d68" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 40403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5cec2aa85594c8c87042c8553263b99" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 40247 } health_report { overall_health: UNKNOWN } } }
Aug 06 21:11:31 dist-test-slave-lgr5 krb5kdc[17912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.178.1@KRBTEST.COM
I20260806 21:11:31.782989 18680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:31.773920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47204 (local address 127.16.178.1:40247)
0806 21:11:31.774051 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:31.774054 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:31.774204 (+ 150us) server_negotiation.cc:407] Connection header received
0806 21:11:31.774329 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:31.774332 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:31.774386 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:31.774473 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:31.775268 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:31.775824 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:31.776779 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:31.777031 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:31.777132 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:31.779760 (+ 2628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:31.779777 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:31.779794 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:31.779859 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:31.781397 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:31.781839 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:31.781844 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:31.781845 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:31.781926 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:31.782124 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:31.782126 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:31.782127 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:31.782384 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:31.782552 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:31.782742 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 21:11:31.782852 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
Aug 06 21:11:31 dist-test-slave-lgr5 krb5kdc[17912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050691, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 21:11:31 dist-test-slave-lgr5 krb5kdc[17912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050691, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.178.1@KRBTEST.COM
I20260806 21:11:31.810456 18680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:31.802406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47214 (local address 127.16.178.1:40247)
0806 21:11:31.802568 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:31.802572 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:31.802678 (+ 106us) server_negotiation.cc:407] Connection header received
0806 21:11:31.802816 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:31.802819 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:31.802865 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:31.802927 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:31.803548 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:31.804052 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:31.804956 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:31.805177 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:31.805267 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:31.807559 (+ 2292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:31.807562 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:31.807564 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:31.807589 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:31.808757 (+ 1168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:31.809159 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:31.809162 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:31.809164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:31.809241 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:31.809493 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:31.809497 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:31.809499 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:31.809678 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:31.809817 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:31.810161 (+ 344us) server_negotiation.cc:299] Negotiation successful
0806 21:11:31.810290 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
Aug 06 21:11:31 dist-test-slave-lgr5 krb5kdc[17912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 21:11:31 dist-test-slave-lgr5 krb5kdc[17912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.178.1@KRBTEST.COM
I20260806 21:11:31.841207 18680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:31.831685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47226 (local address 127.16.178.1:40247)
0806 21:11:31.831831 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:31.831836 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:31.832018 (+ 182us) server_negotiation.cc:407] Connection header received
0806 21:11:31.832170 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:31.832173 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:31.832257 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:31.832362 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:31.833127 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:31.833867 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:31.834751 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:31.835124 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:31.835228 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:31.837885 (+ 2657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:31.837889 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:31.837891 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:31.837925 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:31.839360 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:31.839862 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:31.839865 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:31.839866 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:31.839943 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:31.840220 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:31.840224 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:31.840225 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:31.840464 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:31.840572 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:31.840905 (+ 333us) server_negotiation.cc:299] Negotiation successful
0806 21:11:31.841024 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260806 21:11:31.842950 18152 tablet_service.cc:2908] Rejecting scan request for tablet ab0206b3eafe447baa3bfefa07c72b9e: Uninitialized: clean time has not yet been initialized
I20260806 21:11:31.845319 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 18086
I20260806 21:11:31.863119 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 18285
I20260806 21:11:31.877272 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 18469
I20260806 21:11:31.889513 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 17930
2026-08-06T21:11:31Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2525 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:11:31 dist-test-slave-lgr5 krb5kdc[18717](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:11:31 dist-test-slave-lgr5 krb5kdc[18717](info): set up 2 sockets
Aug 06 21:11:31 dist-test-slave-lgr5 krb5kdc[18717](info): commencing operation
krb5kdc: starting...
W20260806 21:11:33.074677 17096 mini_kdc.cc:121] Time spent starting KDC: real 1.157s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Aug 06 21:11:33 dist-test-slave-lgr5 krb5kdc[18717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:11:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:11:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.178.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
I20260806 21:11:33.207653 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:39949
--webserver_interface=127.16.178.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:35945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:39949
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:11:33.323988 18736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:33.324302 18736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:33.324359 18736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:33.328361 18736 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:11:33.328433 18736 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 21:11:33.328456 18736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:33.328480 18736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:33.328510 18736 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:11:33.328539 18736 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:11:33.334122 18736 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:35945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:39949
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:39949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.178.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:33.335387 18736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:33.336557 18736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:33.345283 18746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:33.345296 18743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:33.345472 18736 server_base.cc:1034] running on GCE node
W20260806 21:11:33.345297 18744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:33.346202 18736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:33.347379 18736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:33.348547 18736 hybrid_clock.cc:648] HybridClock initialized: now 1786050693348528 us; error 30 us; skew 500 ppm
Aug 06 21:11:33 dist-test-slave-lgr5 krb5kdc[18717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050693, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:33.351378 18736 init.cc:377] Logged in from keytab as kudu/127.16.178.62@KRBTEST.COM (short username kudu)
I20260806 21:11:33.352491 18736 webserver.cc:460] Webserver started at http://127.16.178.62:37037/ using document root <none> and password file <none>
I20260806 21:11:33.353066 18736 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:33.353137 18736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:33.353346 18736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:33.355504 18736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "bdbe3f02fb6e499ca8be9c1521c8f20f"
format_stamp: "Formatted at 2026-08-06 21:11:33 on dist-test-slave-lgr5"
server_key: "0aafe2f2f1f52227d6004b9ca77c14f0"
server_key_iv: "1907068d036b955ab41ee03bbf122a56"
server_key_version: "encryptionkey@0"
I20260806 21:11:33.356072 18736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "bdbe3f02fb6e499ca8be9c1521c8f20f"
format_stamp: "Formatted at 2026-08-06 21:11:33 on dist-test-slave-lgr5"
server_key: "0aafe2f2f1f52227d6004b9ca77c14f0"
server_key_iv: "1907068d036b955ab41ee03bbf122a56"
server_key_version: "encryptionkey@0"
I20260806 21:11:33.359970 18736 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 21:11:33.362571 18765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:33.363818 18736 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 21:11:33.363950 18736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "bdbe3f02fb6e499ca8be9c1521c8f20f"
format_stamp: "Formatted at 2026-08-06 21:11:33 on dist-test-slave-lgr5"
server_key: "0aafe2f2f1f52227d6004b9ca77c14f0"
server_key_iv: "1907068d036b955ab41ee03bbf122a56"
server_key_version: "encryptionkey@0"
I20260806 21:11:33.364058 18736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:33.377257 18736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:33.380275 18736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:33.380472 18736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:33.388271 18736 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.62:39949
I20260806 21:11:33.388300 18856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.62:39949 every 8 connection(s)
I20260806 21:11:33.389210 18736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:11:33.391978 18857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:33.395304 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 18736
I20260806 21:11:33.395434 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:11:33.395744 17096 external_mini_cluster.cc:1428] Setting key 2085c8d8dbdf080dfc2a61b68d563eda
I20260806 21:11:33.398061 18857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdbe3f02fb6e499ca8be9c1521c8f20f: Bootstrap starting.
I20260806 21:11:33.400303 18857 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bdbe3f02fb6e499ca8be9c1521c8f20f: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:33.401039 18857 log.cc:824] T 00000000000000000000000000000000 P bdbe3f02fb6e499ca8be9c1521c8f20f: Log is configured to *not* fsync() on all Append() calls
I20260806 21:11:33.402966 18857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdbe3f02fb6e499ca8be9c1521c8f20f: No bootstrap required, opened a new log
Aug 06 21:11:33 dist-test-slave-lgr5 krb5kdc[18717](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:11:33.406802 18857 raft_consensus.cc:348] T 00000000000000000000000000000000 P bdbe3f02fb6e499ca8be9c1521c8f20f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdbe3f02fb6e499ca8be9c1521c8f20f" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 39949 } }
I20260806 21:11:33.407054 18857 raft_consensus.cc:374] T 00000000000000000000000000000000 P bdbe3f02fb6e499ca8be9c1521c8f20f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:33.407116 18857 raft_consensus.cc:729] T 00000000000000000000000000000000 P bdbe3f02fb6e499ca8be9c1521c8f20f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdbe3f02fb6e499ca8be9c1521c8f20f, State: Initialized, Role: FOLLOWER
I20260806 21:11:33.407595 18857 consensus_queue.cc:260] T 00000000000000000000000000000000 P bdbe3f02fb6e499ca8be9c1521c8f20f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdbe3f02fb6e499ca8be9c1521c8f20f" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 39949 } }
I20260806 21:11:33.407758 18857 raft_consensus.cc:388] T 00000000000000000000000000000000 P bdbe3f02fb6e499ca8be9c1521c8f20f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:11:33.407833 18857 raft_consensus.cc:482] T 00000000000000000000000000000000 P bdbe3f02fb6e499ca8be9c1521c8f20f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:11:33.407932 18857 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bdbe3f02fb6e499ca8be9c1521c8f20f [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:33.409025 18857 raft_consensus.cc:504] T 00000000000000000000000000000000 P bdbe3f02fb6e499ca8be9c1521c8f20f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdbe3f02fb6e499ca8be9c1521c8f20f" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 39949 } }
I20260806 21:11:33.409364 18857 leader_election.cc:302] T 00000000000000000000000000000000 P bdbe3f02fb6e499ca8be9c1521c8f20f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bdbe3f02fb6e499ca8be9c1521c8f20f; no voters:
I20260806 21:11:33.409668 18857 leader_election.cc:288] T 00000000000000000000000000000000 P bdbe3f02fb6e499ca8be9c1521c8f20f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:11:33.409756 18866 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bdbe3f02fb6e499ca8be9c1521c8f20f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:11:33.409976 18866 raft_consensus.cc:686] T 00000000000000000000000000000000 P bdbe3f02fb6e499ca8be9c1521c8f20f [term 1 LEADER]: Becoming Leader. State: Replica: bdbe3f02fb6e499ca8be9c1521c8f20f, State: Running, Role: LEADER
I20260806 21:11:33.410034 18862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:33.397311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53208 (local address 127.16.178.62:39949)
0806 21:11:33.397733 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:33.397753 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:33.397789 (+ 36us) server_negotiation.cc:407] Connection header received
0806 21:11:33.398425 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:33.398444 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:33.398780 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:33.399079 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:33.399872 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:33.400953 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:33.402017 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:33.402500 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:33.402660 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:33.405270 (+ 2610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:33.405276 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:33.405290 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:33.405318 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:33.406913 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:33.407423 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:33.407430 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:33.407436 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:33.407575 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:33.407880 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:33.407884 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:33.407885 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:33.408390 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:33.408551 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:33.408829 (+ 278us) server_negotiation.cc:299] Negotiation successful
0806 21:11:33.409129 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":123,"threads_started":1}
I20260806 21:11:33.410476 18866 consensus_queue.cc:237] T 00000000000000000000000000000000 P bdbe3f02fb6e499ca8be9c1521c8f20f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdbe3f02fb6e499ca8be9c1521c8f20f" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 39949 } }
I20260806 21:11:33.411078 18857 sys_catalog.cc:564] T 00000000000000000000000000000000 P bdbe3f02fb6e499ca8be9c1521c8f20f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:11:33.412176 18867 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdbe3f02fb6e499ca8be9c1521c8f20f [sys.catalog]: SysCatalogTable state changed. Reason: New leader bdbe3f02fb6e499ca8be9c1521c8f20f. Latest consensus state: current_term: 1 leader_uuid: "bdbe3f02fb6e499ca8be9c1521c8f20f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdbe3f02fb6e499ca8be9c1521c8f20f" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 39949 } } }
I20260806 21:11:33.412313 18867 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdbe3f02fb6e499ca8be9c1521c8f20f [sys.catalog]: This master's current role is: LEADER
I20260806 21:11:33.412714 18869 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdbe3f02fb6e499ca8be9c1521c8f20f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bdbe3f02fb6e499ca8be9c1521c8f20f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdbe3f02fb6e499ca8be9c1521c8f20f" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 39949 } } }
I20260806 21:11:33.412793 18879 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:11:33.412798 18869 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdbe3f02fb6e499ca8be9c1521c8f20f [sys.catalog]: This master's current role is: LEADER
I20260806 21:11:33.415555 18879 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:11:33.421924 18879 catalog_manager.cc:1347] Generated new cluster ID: 519ef1f4fe894dd39e9a5c9fae6510b8
I20260806 21:11:33.422046 18879 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:11:33.441689 18879 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:11:33.443219 18879 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:11:33.451293 18879 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bdbe3f02fb6e499ca8be9c1521c8f20f: Generated new TSK 0
I20260806 21:11:33.452200 18879 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.178.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab.
Entry for principal kudu/127.16.178.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab.
I20260806 21:11:33.491490 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.1:0
--local_ip_for_outbound_sockets=127.16.178.1
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:39949
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab
--principal=kudu/127.16.178.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:35945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:11:33.608203 18900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:33.608476 18900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:33.608538 18900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:33.612605 18900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:33.612677 18900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:33.612735 18900 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 21:11:33.612788 18900 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.1
I20260806 21:11:33.618055 18900 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:35945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab
--principal=kudu/127.16.178.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.178.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.178.62:39949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.1
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:33.619094 18900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:33.620118 18900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:33.629549 18915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:33.629536 18909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:33.629536 18907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:33.629585 18900 server_base.cc:1034] running on GCE node
I20260806 21:11:33.630141 18900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:33.630764 18900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:33.631928 18900 hybrid_clock.cc:648] HybridClock initialized: now 1786050693631909 us; error 30 us; skew 500 ppm
Aug 06 21:11:33 dist-test-slave-lgr5 krb5kdc[18717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050693, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:33.634749 18900 init.cc:377] Logged in from keytab as kudu/127.16.178.1@KRBTEST.COM (short username kudu)
I20260806 21:11:33.635736 18900 webserver.cc:460] Webserver started at http://127.16.178.1:37357/ using document root <none> and password file <none>
I20260806 21:11:33.636371 18900 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:33.636448 18900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:33.636678 18900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:33.638859 18900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-0/data/instance:
uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a"
format_stamp: "Formatted at 2026-08-06 21:11:33 on dist-test-slave-lgr5"
server_key: "ffe4bb8c2341fe8657019b4670a951e1"
server_key_iv: "f58fee800923e204e95a61b0875c55e2"
server_key_version: "encryptionkey@0"
I20260806 21:11:33.639430 18900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance:
uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a"
format_stamp: "Formatted at 2026-08-06 21:11:33 on dist-test-slave-lgr5"
server_key: "ffe4bb8c2341fe8657019b4670a951e1"
server_key_iv: "f58fee800923e204e95a61b0875c55e2"
server_key_version: "encryptionkey@0"
I20260806 21:11:33.643380 18900 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 21:11:33.645959 18926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:33.647218 18900 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 21:11:33.647346 18900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-0/wal
uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a"
format_stamp: "Formatted at 2026-08-06 21:11:33 on dist-test-slave-lgr5"
server_key: "ffe4bb8c2341fe8657019b4670a951e1"
server_key_iv: "f58fee800923e204e95a61b0875c55e2"
server_key_version: "encryptionkey@0"
I20260806 21:11:33.647456 18900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:33.671236 18900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:33.674706 18900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:33.674914 18900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:33.676618 18900 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:11:33.676676 18900 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:33.676733 18900 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:11:33.676760 18900 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 21:11:33 dist-test-slave-lgr5 krb5kdc[18717](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050693, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:11:33.688452 18900 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.1:32823
I20260806 21:11:33.688552 19113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.1:32823 every 8 connection(s)
I20260806 21:11:33.689571 18900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
I20260806 21:11:33.691602 18862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:33.677414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:59445 (local address 127.16.178.62:39949)
0806 21:11:33.677534 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:33.677537 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:33.678697 (+ 1160us) server_negotiation.cc:407] Connection header received
0806 21:11:33.679803 (+ 1106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:33.679809 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:33.679874 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:33.679937 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:33.681679 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:33.682334 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:33.683739 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:33.684183 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:33.684336 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:33.687499 (+ 3163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:33.687504 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:33.687506 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:33.687542 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:33.689249 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:33.689909 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:33.689913 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:33.689914 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:33.690025 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:33.690441 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:33.690444 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:33.690446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:33.690705 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:33.690811 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:33.691326 (+ 515us) server_negotiation.cc:299] Negotiation successful
0806 21:11:33.691466 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 21:11:33.692582 18935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:33.677983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:39949 (local address 127.16.178.1:59445)
0806 21:11:33.678533 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:33.678577 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:33.679523 (+ 946us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:33.680419 (+ 896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:33.680427 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:33.680851 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:33.681423 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:33.681438 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:33.682480 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:33.682484 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:33.683540 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:33.683560 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:33.684423 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:33.684428 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:33.684643 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:33.685336 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:33.685355 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:33.687223 (+ 1868us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:33.689396 (+ 2173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:33.689403 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:33.689418 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:33.689799 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:33.690149 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:33.690152 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:33.690154 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:33.690338 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:33.690826 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:33.690832 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:33.691102 (+ 270us) client_negotiation.cc:769] Sending connection context
0806 21:11:33.691272 (+ 170us) client_negotiation.cc:240] Negotiation successful
0806 21:11:33.691606 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":138,"threads_started":1}
I20260806 21:11:33.693997 19116 heartbeater.cc:347] Connected to a master server at 127.16.178.62:39949
I20260806 21:11:33.694262 19116 heartbeater.cc:464] Registering TS with master...
I20260806 21:11:33.694975 19116 heartbeater.cc:511] Master 127.16.178.62:39949 requested a full tablet report, sending...
I20260806 21:11:33.697084 18793 ts_manager.cc:194] Registered new tserver with Master: eb58cc0eb6a94359b8c1a4a3a8d9ba3a (127.16.178.1:32823)
I20260806 21:11:33.698331 18793 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.1@KRBTEST.COM'} at 127.16.178.1:59445
I20260806 21:11:33.699862 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 18900
I20260806 21:11:33.699990 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance
I20260806 21:11:33.700330 17096 external_mini_cluster.cc:1428] Setting key d5ce91a6096bd4ac7d2bb16c5a837bcb
I20260806 21:11:33.704555 18862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:33.697157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:60271 (local address 127.16.178.62:39949)
0806 21:11:33.697349 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:33.697354 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:33.697547 (+ 193us) server_negotiation.cc:407] Connection header received
0806 21:11:33.697666 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:33.697669 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:33.697714 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:33.697779 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:33.698266 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:33.698989 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:33.699905 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:33.700240 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:33.700333 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:33.701557 (+ 1224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:33.701561 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:33.701564 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:33.701591 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:33.702964 (+ 1373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:33.703369 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:33.703373 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:33.703375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:33.703455 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:33.703711 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:33.703714 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:33.703716 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:33.703902 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:33.703989 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:33.704286 (+ 297us) server_negotiation.cc:299] Negotiation successful
0806 21:11:33.704397 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260806 21:11:33.704569 19126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:33.697042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:39949 (local address 127.16.178.1:60271)
0806 21:11:33.697464 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:33.697477 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:33.697583 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:33.697808 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:33.697811 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:33.698030 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:33.698137 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:33.698144 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:33.699124 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:33.699127 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:33.699783 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:33.699794 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:33.700367 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:33.700371 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:33.700461 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:33.700971 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:33.700986 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:33.701416 (+ 430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:33.703075 (+ 1659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:33.703078 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:33.703080 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:33.703278 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:33.703540 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:33.703543 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:33.703545 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:33.703628 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:33.703995 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:33.703998 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:33.704119 (+ 121us) client_negotiation.cc:769] Sending connection context
0806 21:11:33.704219 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 21:11:33.704369 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":122,"threads_started":1}
WARNING: no policy specified for kudu/127.16.178.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab.
Entry for principal kudu/127.16.178.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab.
I20260806 21:11:33.729127 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.2:0
--local_ip_for_outbound_sockets=127.16.178.2
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:39949
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab
--principal=kudu/127.16.178.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:35945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:11:33.848635 19130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:33.848935 19130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:33.848982 19130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:33.853163 19130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:33.853232 19130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:33.853272 19130 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 21:11:33.853307 19130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.2
I20260806 21:11:33.858747 19130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:35945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab
--principal=kudu/127.16.178.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.178.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.178.62:39949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.2
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:33.859939 19130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:33.861081 19130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:33.871723 19140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:33.871764 19130 server_base.cc:1034] running on GCE node
W20260806 21:11:33.871716 19138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:33.871716 19137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:33.872525 19130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:33.873260 19130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:33.874440 19130 hybrid_clock.cc:648] HybridClock initialized: now 1786050693874420 us; error 36 us; skew 500 ppm
Aug 06 21:11:33 dist-test-slave-lgr5 krb5kdc[18717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050693, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:33.877292 19130 init.cc:377] Logged in from keytab as kudu/127.16.178.2@KRBTEST.COM (short username kudu)
I20260806 21:11:33.878235 19130 webserver.cc:460] Webserver started at http://127.16.178.2:40845/ using document root <none> and password file <none>
I20260806 21:11:33.878791 19130 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:33.878852 19130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:33.879101 19130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:33.881263 19130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-1/data/instance:
uuid: "892d55ac7316407a9abe214a09505c40"
format_stamp: "Formatted at 2026-08-06 21:11:33 on dist-test-slave-lgr5"
server_key: "167e9d45356a13f2ac3a57e5c3243ab2"
server_key_iv: "dda31a145ef9c6a61e8892004f5d5da2"
server_key_version: "encryptionkey@0"
I20260806 21:11:33.881799 19130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance:
uuid: "892d55ac7316407a9abe214a09505c40"
format_stamp: "Formatted at 2026-08-06 21:11:33 on dist-test-slave-lgr5"
server_key: "167e9d45356a13f2ac3a57e5c3243ab2"
server_key_iv: "dda31a145ef9c6a61e8892004f5d5da2"
server_key_version: "encryptionkey@0"
I20260806 21:11:33.885785 19130 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260806 21:11:33.888221 19154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:33.889506 19130 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 21:11:33.889614 19130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-1/wal
uuid: "892d55ac7316407a9abe214a09505c40"
format_stamp: "Formatted at 2026-08-06 21:11:33 on dist-test-slave-lgr5"
server_key: "167e9d45356a13f2ac3a57e5c3243ab2"
server_key_iv: "dda31a145ef9c6a61e8892004f5d5da2"
server_key_version: "encryptionkey@0"
I20260806 21:11:33.889753 19130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:33.902120 19130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:33.905561 19130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:33.905748 19130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:33.907282 19130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:11:33.907332 19130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:33.907377 19130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:11:33.907413 19130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:33.917495 19130 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.2:36933
Aug 06 21:11:33 dist-test-slave-lgr5 krb5kdc[18717](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050693, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.2@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:11:33.917538 19330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.2:36933 every 8 connection(s)
I20260806 21:11:33.918419 19130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
I20260806 21:11:33.921675 18862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:33.908153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:50423 (local address 127.16.178.62:39949)
0806 21:11:33.908309 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:33.908315 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:33.909161 (+ 846us) server_negotiation.cc:407] Connection header received
0806 21:11:33.910365 (+ 1204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:33.910369 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:33.910433 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:33.910537 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:33.912337 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:33.913219 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:33.914945 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:33.915174 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:33.915253 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:33.918164 (+ 2911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:33.918168 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:33.918171 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:33.918203 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:33.919797 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:33.920353 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:33.920355 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:33.920357 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:33.920468 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:33.920748 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:33.920750 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:33.920751 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:33.920929 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:33.921001 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:33.921450 (+ 449us) server_negotiation.cc:299] Negotiation successful
0806 21:11:33.921554 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 21:11:33.922446 19168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:33.908509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:39949 (local address 127.16.178.2:50423)
0806 21:11:33.908995 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:33.909044 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:33.910104 (+ 1060us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:33.910855 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:33.910863 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:33.911376 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:33.912150 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:33.912173 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:33.913365 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:33.913370 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:33.914798 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:33.914809 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:33.915337 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:33.915342 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:33.915551 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:33.916265 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:33.916286 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:33.917982 (+ 1696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:33.919920 (+ 1938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:33.919924 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:33.919935 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:33.920267 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:33.920542 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:33.920544 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:33.920545 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:33.920676 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:33.921000 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:33.921006 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:33.921234 (+ 228us) client_negotiation.cc:769] Sending connection context
0806 21:11:33.921380 (+ 146us) client_negotiation.cc:240] Negotiation successful
0806 21:11:33.921633 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":138,"threads_started":1}
I20260806 21:11:33.923717 19331 heartbeater.cc:347] Connected to a master server at 127.16.178.62:39949
I20260806 21:11:33.923863 19331 heartbeater.cc:464] Registering TS with master...
I20260806 21:11:33.924463 19331 heartbeater.cc:511] Master 127.16.178.62:39949 requested a full tablet report, sending...
I20260806 21:11:33.925632 18793 ts_manager.cc:194] Registered new tserver with Master: 892d55ac7316407a9abe214a09505c40 (127.16.178.2:36933)
I20260806 21:11:33.926343 18793 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.2@KRBTEST.COM'} at 127.16.178.2:50423
I20260806 21:11:33.926658 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 19130
I20260806 21:11:33.926767 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance
I20260806 21:11:33.927086 17096 external_mini_cluster.cc:1428] Setting key 3c54b76f1f4039d886107dcfe90e1098
I20260806 21:11:33.933322 19341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:33.926068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:39949 (local address 127.16.178.2:51855)
0806 21:11:33.926707 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:33.926721 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:33.926856 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:33.927105 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:33.927107 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:33.927295 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:33.927389 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:33.927395 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:33.928404 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:33.928408 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:33.928971 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:33.928980 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:33.929485 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:33.929487 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:33.929555 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:33.930014 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:33.930024 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:33.930324 (+ 300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:33.931766 (+ 1442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:33.931769 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:33.931771 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:33.931965 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:33.932244 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:33.932247 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:33.932249 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:33.932398 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:33.932805 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:33.932808 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:33.932894 (+ 86us) client_negotiation.cc:769] Sending connection context
0806 21:11:33.932990 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 21:11:33.933132 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":117,"threads_started":1}
I20260806 21:11:33.933317 18862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:33.926170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:51855 (local address 127.16.178.62:39949)
0806 21:11:33.926291 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:33.926295 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:33.926789 (+ 494us) server_negotiation.cc:407] Connection header received
0806 21:11:33.926970 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:33.926973 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:33.927015 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:33.927078 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:33.927509 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:33.928288 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:33.929090 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:33.929376 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:33.929490 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:33.930463 (+ 973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:33.930467 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:33.930469 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:33.930493 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:33.931664 (+ 1171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:33.932052 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:33.932055 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:33.932057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:33.932161 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:33.932490 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:33.932493 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:33.932495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:33.932712 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:33.932799 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:33.933056 (+ 257us) server_negotiation.cc:299] Negotiation successful
0806 21:11:33.933160 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
WARNING: no policy specified for kudu/127.16.178.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab.
Entry for principal kudu/127.16.178.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab.
I20260806 21:11:33.955312 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.3:0
--local_ip_for_outbound_sockets=127.16.178.3
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:39949
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab
--principal=kudu/127.16.178.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:35945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:11:34.078351 19345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:34.078672 19345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:34.078717 19345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:34.082865 19345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:34.082939 19345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:34.082978 19345 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 21:11:34.083020 19345 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.3
I20260806 21:11:34.088413 19345 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:35945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab
--principal=kudu/127.16.178.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.178.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.178.62:39949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.3
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:34.089502 19345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:34.090510 19345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:34.100329 19351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:34.100364 19356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:34.100425 19352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:34.100488 19345 server_base.cc:1034] running on GCE node
I20260806 21:11:34.100965 19345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:34.101593 19345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:34.102777 19345 hybrid_clock.cc:648] HybridClock initialized: now 1786050694102756 us; error 29 us; skew 500 ppm
Aug 06 21:11:34 dist-test-slave-lgr5 krb5kdc[18717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050694, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:34.105518 19345 init.cc:377] Logged in from keytab as kudu/127.16.178.3@KRBTEST.COM (short username kudu)
I20260806 21:11:34.106387 19345 webserver.cc:460] Webserver started at http://127.16.178.3:42605/ using document root <none> and password file <none>
I20260806 21:11:34.106938 19345 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:34.107000 19345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:34.107196 19345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:34.109272 19345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-2/data/instance:
uuid: "5311aac7a92645e3967b76b74515ca20"
format_stamp: "Formatted at 2026-08-06 21:11:34 on dist-test-slave-lgr5"
server_key: "1c32d3eb5f28454827f89226e652f8cf"
server_key_iv: "a72156f6ef59e8dbdd4258481c538a3e"
server_key_version: "encryptionkey@0"
I20260806 21:11:34.109823 19345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance:
uuid: "5311aac7a92645e3967b76b74515ca20"
format_stamp: "Formatted at 2026-08-06 21:11:34 on dist-test-slave-lgr5"
server_key: "1c32d3eb5f28454827f89226e652f8cf"
server_key_iv: "a72156f6ef59e8dbdd4258481c538a3e"
server_key_version: "encryptionkey@0"
I20260806 21:11:34.113711 19345 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:11:34.116163 19368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:34.117354 19345 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 21:11:34.117463 19345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-2/wal
uuid: "5311aac7a92645e3967b76b74515ca20"
format_stamp: "Formatted at 2026-08-06 21:11:34 on dist-test-slave-lgr5"
server_key: "1c32d3eb5f28454827f89226e652f8cf"
server_key_iv: "a72156f6ef59e8dbdd4258481c538a3e"
server_key_version: "encryptionkey@0"
I20260806 21:11:34.117571 19345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:34.131922 19345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:34.135316 19345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:34.135500 19345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:34.137099 19345 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:11:34.137161 19345 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:34.137217 19345 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:11:34.137244 19345 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 21:11:34 dist-test-slave-lgr5 krb5kdc[18717](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050694, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.3@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:11:34.149391 19345 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.3:38633
I20260806 21:11:34.149483 19540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.3:38633 every 8 connection(s)
I20260806 21:11:34.150501 19345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
I20260806 21:11:34.153733 18862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:34.137974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:46825 (local address 127.16.178.62:39949)
0806 21:11:34.138099 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:34.138103 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:34.139255 (+ 1152us) server_negotiation.cc:407] Connection header received
0806 21:11:34.140748 (+ 1493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:34.140752 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:34.140812 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:34.140896 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:34.142650 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:34.143533 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:34.146137 (+ 2604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:34.146480 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:34.146618 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:34.149733 (+ 3115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:34.149737 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:34.149739 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:34.149770 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:34.151454 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:34.152032 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:34.152036 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:34.152038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:34.152160 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:34.152527 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:34.152531 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:34.152533 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:34.152750 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:34.152852 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:34.153400 (+ 548us) server_negotiation.cc:299] Negotiation successful
0806 21:11:34.153540 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 21:11:34.154088 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 19345
I20260806 21:11:34.154206 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance
I20260806 21:11:34.154507 17096 external_mini_cluster.cc:1428] Setting key 3618f9c175026f620dd2b80ccc78d2e5
I20260806 21:11:34.154646 19380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:34.138523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:39949 (local address 127.16.178.3:46825)
0806 21:11:34.139095 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:34.139155 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:34.140384 (+ 1229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:34.141218 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:34.141228 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:34.141719 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:34.142454 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:34.142479 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:34.143784 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:34.143788 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:34.145888 (+ 2100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:34.145908 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:34.146702 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:34.146706 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:34.146904 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:34.147669 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:34.147688 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:34.149479 (+ 1791us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:34.151589 (+ 2110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:34.151596 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:34.151613 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:34.151906 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:34.152251 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:34.152255 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:34.152257 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:34.152414 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:34.152863 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:34.152872 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:34.153131 (+ 259us) client_negotiation.cc:769] Sending connection context
0806 21:11:34.153309 (+ 178us) client_negotiation.cc:240] Negotiation successful
0806 21:11:34.153688 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":137,"threads_started":1}
I20260806 21:11:34.155858 19541 heartbeater.cc:347] Connected to a master server at 127.16.178.62:39949
I20260806 21:11:34.156015 19541 heartbeater.cc:464] Registering TS with master...
I20260806 21:11:34.156653 19541 heartbeater.cc:511] Master 127.16.178.62:39949 requested a full tablet report, sending...
I20260806 21:11:34.157763 18788 ts_manager.cc:194] Registered new tserver with Master: 5311aac7a92645e3967b76b74515ca20 (127.16.178.3:38633)
I20260806 21:11:34.158434 18788 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.3@KRBTEST.COM'} at 127.16.178.3:46825
I20260806 21:11:34.165336 18862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:34.158146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:56753 (local address 127.16.178.62:39949)
0806 21:11:34.158279 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:34.158285 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:34.158912 (+ 627us) server_negotiation.cc:407] Connection header received
0806 21:11:34.158980 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:34.158983 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:34.159035 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:34.159105 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:34.159624 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:34.160199 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:34.161126 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:34.161343 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:34.161425 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:34.162525 (+ 1100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:34.162529 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:34.162531 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:34.162556 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:34.163897 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:34.164339 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:34.164341 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:34.164343 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:34.164423 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:34.164623 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:34.164625 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:34.164626 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:34.164809 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:34.164881 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:34.165102 (+ 221us) server_negotiation.cc:299] Negotiation successful
0806 21:11:34.165205 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 21:11:34.165369 19550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:34.158060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:39949 (local address 127.16.178.3:56753)
0806 21:11:34.158811 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:34.158826 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:34.158911 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:34.159138 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:34.159141 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:34.159371 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:34.159487 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:34.159496 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:34.160330 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:34.160334 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:34.161019 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:34.161030 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:34.161435 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:34.161437 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:34.161513 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:34.162032 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:34.162044 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:34.162386 (+ 342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:34.164004 (+ 1618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:34.164007 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:34.164008 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:34.164255 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:34.164488 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:34.164490 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:34.164492 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:34.164566 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:34.164872 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:34.164874 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:34.164979 (+ 105us) client_negotiation.cc:769] Sending connection context
0806 21:11:34.165059 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 21:11:34.165194 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":661,"thread_start_us":111,"threads_started":1}
I20260806 21:11:34.168901 17096 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 21:11:34 dist-test-slave-lgr5 krb5kdc[18717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050694, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 21:11:34 dist-test-slave-lgr5 krb5kdc[18717](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050694, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:11:34.196532 18862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:34.187727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53222 (local address 127.16.178.62:39949)
0806 21:11:34.187873 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:34.187877 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:34.187892 (+ 15us) server_negotiation.cc:407] Connection header received
0806 21:11:34.187936 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:34.187938 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:34.187991 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:34.188063 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:34.188873 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:34.189425 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:34.190258 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:34.190486 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:34.190579 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:34.193117 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:34.193121 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:34.193123 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:34.193154 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:34.194739 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:34.195217 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:34.195221 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:34.195223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:34.195312 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:34.195607 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:34.195611 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:34.195613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:34.195835 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:34.195961 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:34.196238 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 21:11:34.196363 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 21:11:34.200034 18788 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53222:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:11:34.203186 18788 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:11:34.219408 19567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:34.213261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55054 (local address 127.16.178.3:38633)
0806 21:11:34.214064 (+ 803us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:34.214077 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:34.214241 (+ 164us) server_negotiation.cc:407] Connection header received
0806 21:11:34.214471 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:34.214477 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:34.214698 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:34.214845 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:11:34.215134 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:34.216361 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:34.217957 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:34.218717 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:34.218829 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:34.219017 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:34.219134 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 21:11:34.219247 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":692,"thread_start_us":151,"threads_started":1}
I20260806 21:11:34.219403 19568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:34.213090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.3:38633 (local address 127.0.0.1:55054)
0806 21:11:34.214094 (+ 1004us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:34.214153 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:34.214312 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:34.214855 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:34.214861 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:34.214884 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:11:34.215038 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:34.215047 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:34.216495 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:34.216498 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:34.217832 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:34.217843 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:34.218855 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:34.218858 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:34.218944 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:34.218980 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 21:11:34.219095 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 21:11:34.219224 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":761,"mutex_wait_us":76,"thread_start_us":113,"threads_started":1}
I20260806 21:11:34.219762 19572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:34.214370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56274 (local address 127.16.178.2:36933)
0806 21:11:34.214803 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:34.214815 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:34.214837 (+ 22us) server_negotiation.cc:407] Connection header received
0806 21:11:34.214913 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:34.214917 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:34.215085 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:34.215233 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:11:34.215459 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:34.216635 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:34.218238 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:34.219045 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:34.219150 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:34.219297 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:34.219501 (+ 204us) server_negotiation.cc:299] Negotiation successful
0806 21:11:34.219619 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":130,"threads_started":1}
I20260806 21:11:34.219851 19565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:34.213937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.2:36933 (local address 127.0.0.1:56274)
0806 21:11:34.214514 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:34.214527 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:34.214597 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:34.215216 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:34.215218 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:34.215238 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:11:34.215321 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:34.215326 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:34.216754 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:34.216757 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:34.218114 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:34.218121 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:34.219171 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:34.219174 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:34.219253 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:34.219275 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 21:11:34.219354 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 21:11:34.219417 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"mutex_wait_us":78,"thread_start_us":92,"threads_started":1}
I20260806 21:11:34.222535 19225 tablet_service.cc:1467] Processing CreateTablet for tablet 2b66f52ba6814a4d96f062c9c98a30c4 (DEFAULT_TABLE table=test-table [id=3edced899a2e4960b6b27726ebf96a4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:11:34.222535 19418 tablet_service.cc:1467] Processing CreateTablet for tablet 2b66f52ba6814a4d96f062c9c98a30c4 (DEFAULT_TABLE table=test-table [id=3edced899a2e4960b6b27726ebf96a4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:11:34.223568 19225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b66f52ba6814a4d96f062c9c98a30c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:34.223575 19418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b66f52ba6814a4d96f062c9c98a30c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:34.229753 19571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:34.213325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:32823 (local address 127.0.0.1:43620)
0806 21:11:34.214270 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:34.214286 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:34.214374 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:34.215404 (+ 1030us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:34.215407 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:34.215425 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:11:34.215520 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:34.215527 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:34.216585 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:34.216589 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:34.218106 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:34.218118 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:34.229244 (+ 11126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:34.229249 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:34.229364 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:34.229396 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 21:11:34.229476 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 21:11:34.229592 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":824,"mutex_wait_us":54,"thread_start_us":104,"threads_started":1}
I20260806 21:11:34.230659 19569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:34.213590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43620 (local address 127.16.178.1:32823)
0806 21:11:34.214819 (+ 1229us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:34.214826 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:34.214846 (+ 20us) server_negotiation.cc:407] Connection header received
0806 21:11:34.215117 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:34.215122 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:34.215292 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:34.215422 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:11:34.215633 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:34.216461 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:34.219368 (+ 2907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:34.229066 (+ 9698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:34.229203 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:34.230348 (+ 1145us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:34.230412 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 21:11:34.230524 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1133,"thread_start_us":77,"threads_started":1}
I20260806 21:11:34.231225 19573 tablet_bootstrap.cc:492] T 2b66f52ba6814a4d96f062c9c98a30c4 P 892d55ac7316407a9abe214a09505c40: Bootstrap starting.
I20260806 21:11:34.233314 19575 tablet_bootstrap.cc:492] T 2b66f52ba6814a4d96f062c9c98a30c4 P 5311aac7a92645e3967b76b74515ca20: Bootstrap starting.
I20260806 21:11:34.233342 19573 tablet_bootstrap.cc:654] T 2b66f52ba6814a4d96f062c9c98a30c4 P 892d55ac7316407a9abe214a09505c40: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:34.233811 19002 tablet_service.cc:1467] Processing CreateTablet for tablet 2b66f52ba6814a4d96f062c9c98a30c4 (DEFAULT_TABLE table=test-table [id=3edced899a2e4960b6b27726ebf96a4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:11:34.234675 19573 log.cc:824] T 2b66f52ba6814a4d96f062c9c98a30c4 P 892d55ac7316407a9abe214a09505c40: Log is configured to *not* fsync() on all Append() calls
I20260806 21:11:34.234899 19002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b66f52ba6814a4d96f062c9c98a30c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:34.235531 19575 tablet_bootstrap.cc:654] T 2b66f52ba6814a4d96f062c9c98a30c4 P 5311aac7a92645e3967b76b74515ca20: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:34.236407 19575 log.cc:824] T 2b66f52ba6814a4d96f062c9c98a30c4 P 5311aac7a92645e3967b76b74515ca20: Log is configured to *not* fsync() on all Append() calls
I20260806 21:11:34.237339 19573 tablet_bootstrap.cc:492] T 2b66f52ba6814a4d96f062c9c98a30c4 P 892d55ac7316407a9abe214a09505c40: No bootstrap required, opened a new log
I20260806 21:11:34.237548 19573 ts_tablet_manager.cc:1397] T 2b66f52ba6814a4d96f062c9c98a30c4 P 892d55ac7316407a9abe214a09505c40: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 21:11:34.238400 19575 tablet_bootstrap.cc:492] T 2b66f52ba6814a4d96f062c9c98a30c4 P 5311aac7a92645e3967b76b74515ca20: No bootstrap required, opened a new log
I20260806 21:11:34.238603 19575 ts_tablet_manager.cc:1397] T 2b66f52ba6814a4d96f062c9c98a30c4 P 5311aac7a92645e3967b76b74515ca20: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 21:11:34.241497 19573 raft_consensus.cc:348] T 2b66f52ba6814a4d96f062c9c98a30c4 P 892d55ac7316407a9abe214a09505c40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5311aac7a92645e3967b76b74515ca20" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 38633 } } peers { permanent_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32823 } } peers { permanent_uuid: "892d55ac7316407a9abe214a09505c40" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 36933 } }
I20260806 21:11:34.241786 19573 raft_consensus.cc:374] T 2b66f52ba6814a4d96f062c9c98a30c4 P 892d55ac7316407a9abe214a09505c40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:34.241839 19573 raft_consensus.cc:729] T 2b66f52ba6814a4d96f062c9c98a30c4 P 892d55ac7316407a9abe214a09505c40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 892d55ac7316407a9abe214a09505c40, State: Initialized, Role: FOLLOWER
I20260806 21:11:34.241750 19582 tablet_bootstrap.cc:492] T 2b66f52ba6814a4d96f062c9c98a30c4 P eb58cc0eb6a94359b8c1a4a3a8d9ba3a: Bootstrap starting.
I20260806 21:11:34.242386 19575 raft_consensus.cc:348] T 2b66f52ba6814a4d96f062c9c98a30c4 P 5311aac7a92645e3967b76b74515ca20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5311aac7a92645e3967b76b74515ca20" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 38633 } } peers { permanent_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32823 } } peers { permanent_uuid: "892d55ac7316407a9abe214a09505c40" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 36933 } }
I20260806 21:11:34.242674 19575 raft_consensus.cc:374] T 2b66f52ba6814a4d96f062c9c98a30c4 P 5311aac7a92645e3967b76b74515ca20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:34.242740 19575 raft_consensus.cc:729] T 2b66f52ba6814a4d96f062c9c98a30c4 P 5311aac7a92645e3967b76b74515ca20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5311aac7a92645e3967b76b74515ca20, State: Initialized, Role: FOLLOWER
I20260806 21:11:34.242682 19573 consensus_queue.cc:260] T 2b66f52ba6814a4d96f062c9c98a30c4 P 892d55ac7316407a9abe214a09505c40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5311aac7a92645e3967b76b74515ca20" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 38633 } } peers { permanent_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32823 } } peers { permanent_uuid: "892d55ac7316407a9abe214a09505c40" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 36933 } }
I20260806 21:11:34.243177 19575 consensus_queue.cc:260] T 2b66f52ba6814a4d96f062c9c98a30c4 P 5311aac7a92645e3967b76b74515ca20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5311aac7a92645e3967b76b74515ca20" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 38633 } } peers { permanent_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32823 } } peers { permanent_uuid: "892d55ac7316407a9abe214a09505c40" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 36933 } }
I20260806 21:11:34.243606 19331 heartbeater.cc:503] Master 127.16.178.62:39949 was elected leader, sending a full tablet report...
I20260806 21:11:34.243726 19573 ts_tablet_manager.cc:1428] T 2b66f52ba6814a4d96f062c9c98a30c4 P 892d55ac7316407a9abe214a09505c40: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 21:11:34.243821 19541 heartbeater.cc:503] Master 127.16.178.62:39949 was elected leader, sending a full tablet report...
I20260806 21:11:34.245072 19575 ts_tablet_manager.cc:1428] T 2b66f52ba6814a4d96f062c9c98a30c4 P 5311aac7a92645e3967b76b74515ca20: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 21:11:34.245247 19582 tablet_bootstrap.cc:654] T 2b66f52ba6814a4d96f062c9c98a30c4 P eb58cc0eb6a94359b8c1a4a3a8d9ba3a: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:34.245940 19582 log.cc:824] T 2b66f52ba6814a4d96f062c9c98a30c4 P eb58cc0eb6a94359b8c1a4a3a8d9ba3a: Log is configured to *not* fsync() on all Append() calls
I20260806 21:11:34.248042 19582 tablet_bootstrap.cc:492] T 2b66f52ba6814a4d96f062c9c98a30c4 P eb58cc0eb6a94359b8c1a4a3a8d9ba3a: No bootstrap required, opened a new log
I20260806 21:11:34.248251 19582 ts_tablet_manager.cc:1397] T 2b66f52ba6814a4d96f062c9c98a30c4 P eb58cc0eb6a94359b8c1a4a3a8d9ba3a: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.001s
I20260806 21:11:34.251672 19582 raft_consensus.cc:348] T 2b66f52ba6814a4d96f062c9c98a30c4 P eb58cc0eb6a94359b8c1a4a3a8d9ba3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5311aac7a92645e3967b76b74515ca20" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 38633 } } peers { permanent_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32823 } } peers { permanent_uuid: "892d55ac7316407a9abe214a09505c40" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 36933 } }
I20260806 21:11:34.251895 19582 raft_consensus.cc:374] T 2b66f52ba6814a4d96f062c9c98a30c4 P eb58cc0eb6a94359b8c1a4a3a8d9ba3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:34.251943 19582 raft_consensus.cc:729] T 2b66f52ba6814a4d96f062c9c98a30c4 P eb58cc0eb6a94359b8c1a4a3a8d9ba3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb58cc0eb6a94359b8c1a4a3a8d9ba3a, State: Initialized, Role: FOLLOWER
I20260806 21:11:34.252365 19582 consensus_queue.cc:260] T 2b66f52ba6814a4d96f062c9c98a30c4 P eb58cc0eb6a94359b8c1a4a3a8d9ba3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5311aac7a92645e3967b76b74515ca20" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 38633 } } peers { permanent_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32823 } } peers { permanent_uuid: "892d55ac7316407a9abe214a09505c40" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 36933 } }
I20260806 21:11:34.253060 19116 heartbeater.cc:503] Master 127.16.178.62:39949 was elected leader, sending a full tablet report...
I20260806 21:11:34.253328 19582 ts_tablet_manager.cc:1428] T 2b66f52ba6814a4d96f062c9c98a30c4 P eb58cc0eb6a94359b8c1a4a3a8d9ba3a: Time spent starting tablet: real 0.005s user 0.003s sys 0.002s
I20260806 21:11:34.395689 19583 raft_consensus.cc:482] T 2b66f52ba6814a4d96f062c9c98a30c4 P 892d55ac7316407a9abe214a09505c40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:11:34.395932 19583 raft_consensus.cc:504] T 2b66f52ba6814a4d96f062c9c98a30c4 P 892d55ac7316407a9abe214a09505c40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5311aac7a92645e3967b76b74515ca20" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 38633 } } peers { permanent_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32823 } } peers { permanent_uuid: "892d55ac7316407a9abe214a09505c40" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 36933 } }
I20260806 21:11:34.397143 19583 leader_election.cc:288] T 2b66f52ba6814a4d96f062c9c98a30c4 P 892d55ac7316407a9abe214a09505c40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5311aac7a92645e3967b76b74515ca20 (127.16.178.3:38633), eb58cc0eb6a94359b8c1a4a3a8d9ba3a (127.16.178.1:32823)
I20260806 21:11:34.401484 19168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:34.397448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.3:38633 (local address 127.16.178.2:44985)
0806 21:11:34.397662 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:34.397678 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:34.397782 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:34.398080 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:34.398083 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:34.398141 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:11:34.398263 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:34.398269 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:34.399093 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:34.399096 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:34.400190 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:34.400203 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:34.401022 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:34.401026 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:34.401174 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:34.401202 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 21:11:34.401271 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 21:11:34.401339 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":25,"spinlock_wait_cycles":20224}
I20260806 21:11:34.401645 19567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:34.397551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:44985 (local address 127.16.178.3:38633)
0806 21:11:34.397793 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:34.397798 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:34.397856 (+ 58us) server_negotiation.cc:407] Connection header received
0806 21:11:34.397921 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:34.397924 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:34.397974 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:34.398045 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:11:34.398400 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:34.398990 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:34.400319 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:34.400897 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:34.401024 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:34.401082 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:34.401443 (+ 361us) server_negotiation.cc:299] Negotiation successful
0806 21:11:34.401526 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149}
W20260806 21:11:34.401846 19542 tablet.cc:2369] T 2b66f52ba6814a4d96f062c9c98a30c4 P 5311aac7a92645e3967b76b74515ca20: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 21:11:34.402268 19469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b66f52ba6814a4d96f062c9c98a30c4" candidate_uuid: "892d55ac7316407a9abe214a09505c40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5311aac7a92645e3967b76b74515ca20" is_pre_election: true
I20260806 21:11:34.402560 19469 raft_consensus.cc:2454] T 2b66f52ba6814a4d96f062c9c98a30c4 P 5311aac7a92645e3967b76b74515ca20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 892d55ac7316407a9abe214a09505c40 in term 0.
I20260806 21:11:34.402769 19591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:34.397448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:32823 (local address 127.16.178.2:38087)
0806 21:11:34.397958 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:34.397971 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:34.398104 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:34.398394 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:34.398396 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:34.398415 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:11:34.398532 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:34.398538 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:34.399501 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:34.399504 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:34.401451 (+ 1947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:34.401460 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:34.402351 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:34.402354 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:34.402459 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:34.402483 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 21:11:34.402558 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 21:11:34.402624 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"mutex_wait_us":179,"thread_start_us":89,"threads_started":1}
I20260806 21:11:34.402786 19569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:34.397642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:38087 (local address 127.16.178.1:32823)
0806 21:11:34.397954 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:34.397958 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:34.398041 (+ 83us) server_negotiation.cc:407] Connection header received
0806 21:11:34.398220 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:34.398223 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:34.398268 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:34.398336 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:11:34.398632 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:34.399381 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:34.401586 (+ 2205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:34.402191 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:34.402323 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:34.402373 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:34.402593 (+ 220us) server_negotiation.cc:299] Negotiation successful
0806 21:11:34.402672 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226}
I20260806 21:11:34.403064 19158 leader_election.cc:302] T 2b66f52ba6814a4d96f062c9c98a30c4 P 892d55ac7316407a9abe214a09505c40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5311aac7a92645e3967b76b74515ca20, 892d55ac7316407a9abe214a09505c40; no voters:
I20260806 21:11:34.403337 19583 raft_consensus.cc:2781] T 2b66f52ba6814a4d96f062c9c98a30c4 P 892d55ac7316407a9abe214a09505c40 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:11:34.403419 19583 raft_consensus.cc:482] T 2b66f52ba6814a4d96f062c9c98a30c4 P 892d55ac7316407a9abe214a09505c40 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:11:34.403450 19583 raft_consensus.cc:3037] T 2b66f52ba6814a4d96f062c9c98a30c4 P 892d55ac7316407a9abe214a09505c40 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:34.403710 19039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b66f52ba6814a4d96f062c9c98a30c4" candidate_uuid: "892d55ac7316407a9abe214a09505c40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a" is_pre_election: true
I20260806 21:11:34.403961 19039 raft_consensus.cc:2454] T 2b66f52ba6814a4d96f062c9c98a30c4 P eb58cc0eb6a94359b8c1a4a3a8d9ba3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 892d55ac7316407a9abe214a09505c40 in term 0.
I20260806 21:11:34.404667 19583 raft_consensus.cc:504] T 2b66f52ba6814a4d96f062c9c98a30c4 P 892d55ac7316407a9abe214a09505c40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5311aac7a92645e3967b76b74515ca20" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 38633 } } peers { permanent_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32823 } } peers { permanent_uuid: "892d55ac7316407a9abe214a09505c40" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 36933 } }
I20260806 21:11:34.405000 19583 leader_election.cc:288] T 2b66f52ba6814a4d96f062c9c98a30c4 P 892d55ac7316407a9abe214a09505c40 [CANDIDATE]: Term 1 election: Requested vote from peers 5311aac7a92645e3967b76b74515ca20 (127.16.178.3:38633), eb58cc0eb6a94359b8c1a4a3a8d9ba3a (127.16.178.1:32823)
I20260806 21:11:34.405421 19469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b66f52ba6814a4d96f062c9c98a30c4" candidate_uuid: "892d55ac7316407a9abe214a09505c40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5311aac7a92645e3967b76b74515ca20"
I20260806 21:11:34.405467 19039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b66f52ba6814a4d96f062c9c98a30c4" candidate_uuid: "892d55ac7316407a9abe214a09505c40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a"
I20260806 21:11:34.405563 19469 raft_consensus.cc:3037] T 2b66f52ba6814a4d96f062c9c98a30c4 P 5311aac7a92645e3967b76b74515ca20 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:34.405563 19039 raft_consensus.cc:3037] T 2b66f52ba6814a4d96f062c9c98a30c4 P eb58cc0eb6a94359b8c1a4a3a8d9ba3a [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:34.406872 19039 raft_consensus.cc:2454] T 2b66f52ba6814a4d96f062c9c98a30c4 P eb58cc0eb6a94359b8c1a4a3a8d9ba3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 892d55ac7316407a9abe214a09505c40 in term 1.
I20260806 21:11:34.407219 19155 leader_election.cc:302] T 2b66f52ba6814a4d96f062c9c98a30c4 P 892d55ac7316407a9abe214a09505c40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 892d55ac7316407a9abe214a09505c40, eb58cc0eb6a94359b8c1a4a3a8d9ba3a; no voters:
I20260806 21:11:34.407356 19469 raft_consensus.cc:2454] T 2b66f52ba6814a4d96f062c9c98a30c4 P 5311aac7a92645e3967b76b74515ca20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 892d55ac7316407a9abe214a09505c40 in term 1.
I20260806 21:11:34.407398 19583 raft_consensus.cc:2781] T 2b66f52ba6814a4d96f062c9c98a30c4 P 892d55ac7316407a9abe214a09505c40 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:11:34.407612 19583 raft_consensus.cc:686] T 2b66f52ba6814a4d96f062c9c98a30c4 P 892d55ac7316407a9abe214a09505c40 [term 1 LEADER]: Becoming Leader. State: Replica: 892d55ac7316407a9abe214a09505c40, State: Running, Role: LEADER
I20260806 21:11:34.407958 19583 consensus_queue.cc:237] T 2b66f52ba6814a4d96f062c9c98a30c4 P 892d55ac7316407a9abe214a09505c40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5311aac7a92645e3967b76b74515ca20" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 38633 } } peers { permanent_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32823 } } peers { permanent_uuid: "892d55ac7316407a9abe214a09505c40" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 36933 } }
I20260806 21:11:34.411026 18793 catalog_manager.cc:5495] T 2b66f52ba6814a4d96f062c9c98a30c4 P 892d55ac7316407a9abe214a09505c40 reported cstate change: term changed from 0 to 1, leader changed from <none> to 892d55ac7316407a9abe214a09505c40 (127.16.178.2). New cstate: current_term: 1 leader_uuid: "892d55ac7316407a9abe214a09505c40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5311aac7a92645e3967b76b74515ca20" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 38633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "892d55ac7316407a9abe214a09505c40" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 36933 } health_report { overall_health: HEALTHY } } }
W20260806 21:11:34.419859 19332 tablet.cc:2369] T 2b66f52ba6814a4d96f062c9c98a30c4 P 892d55ac7316407a9abe214a09505c40: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 21:11:34.441056 19118 tablet.cc:2369] T 2b66f52ba6814a4d96f062c9c98a30c4 P eb58cc0eb6a94359b8c1a4a3a8d9ba3a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 21:11:34.466703 19572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:34.462786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56280 (local address 127.16.178.2:36933)
0806 21:11:34.462932 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:34.462937 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:34.462955 (+ 18us) server_negotiation.cc:407] Connection header received
0806 21:11:34.463181 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:34.463183 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:34.463250 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:34.463373 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:11:34.463652 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:34.464219 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:34.465147 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:34.465383 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:34.465487 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:34.465880 (+ 393us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:11:34.466345 (+ 465us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:11:34.466438 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:34.466513 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 21:11:34.466572 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 21:11:34.473418 19039 raft_consensus.cc:1261] T 2b66f52ba6814a4d96f062c9c98a30c4 P eb58cc0eb6a94359b8c1a4a3a8d9ba3a [term 1 FOLLOWER]: Refusing update from remote peer 892d55ac7316407a9abe214a09505c40: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:11:34.474167 19583 consensus_queue.cc:1035] T 2b66f52ba6814a4d96f062c9c98a30c4 P 892d55ac7316407a9abe214a09505c40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:11:34.474965 19469 raft_consensus.cc:1261] T 2b66f52ba6814a4d96f062c9c98a30c4 P 5311aac7a92645e3967b76b74515ca20 [term 1 FOLLOWER]: Refusing update from remote peer 892d55ac7316407a9abe214a09505c40: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:11:34.475615 19583 consensus_queue.cc:1035] T 2b66f52ba6814a4d96f062c9c98a30c4 P 892d55ac7316407a9abe214a09505c40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5311aac7a92645e3967b76b74515ca20" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 38633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:11:34.482791 19597 mvcc.cc:205] Tried to move back new op lower bound from 7315663644554387456 to 7315663644297854976. Current Snapshot: MvccSnapshot[applied={T|T < 7315663644554387456}]
I20260806 21:11:34.483268 19599 mvcc.cc:205] Tried to move back new op lower bound from 7315663644554387456 to 7315663644297854976. Current Snapshot: MvccSnapshot[applied={T|T < 7315663644554387456}]
I20260806 21:11:34.496030 18793 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53222:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:11:34.496284 18793 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53222:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:11:34.499056 18793 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bdbe3f02fb6e499ca8be9c1521c8f20f: Sending DeleteTablet for 3 replicas of tablet 2b66f52ba6814a4d96f062c9c98a30c4
I20260806 21:11:34.499945 19418 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b66f52ba6814a4d96f062c9c98a30c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:11:34 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:55054
I20260806 21:11:34.500219 19225 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b66f52ba6814a4d96f062c9c98a30c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:11:34 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:56274
I20260806 21:11:34.500224 19002 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b66f52ba6814a4d96f062c9c98a30c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:11:34 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:43620
I20260806 21:11:34.500699 19609 tablet_replica.cc:330] T 2b66f52ba6814a4d96f062c9c98a30c4 P eb58cc0eb6a94359b8c1a4a3a8d9ba3a: stopping tablet replica
I20260806 21:11:34.500906 19611 tablet_replica.cc:330] T 2b66f52ba6814a4d96f062c9c98a30c4 P 892d55ac7316407a9abe214a09505c40: stopping tablet replica
I20260806 21:11:34.501042 19609 raft_consensus.cc:2229] T 2b66f52ba6814a4d96f062c9c98a30c4 P eb58cc0eb6a94359b8c1a4a3a8d9ba3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:11:34.501279 19611 raft_consensus.cc:2229] T 2b66f52ba6814a4d96f062c9c98a30c4 P 892d55ac7316407a9abe214a09505c40 [term 1 LEADER]: Raft consensus shutting down.
I20260806 21:11:34.501413 19609 raft_consensus.cc:2258] T 2b66f52ba6814a4d96f062c9c98a30c4 P eb58cc0eb6a94359b8c1a4a3a8d9ba3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:11:34.501518 18789 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53222:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 21:11:34.501899 19611 raft_consensus.cc:2258] T 2b66f52ba6814a4d96f062c9c98a30c4 P 892d55ac7316407a9abe214a09505c40 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 21:11:34.501981 18789 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:11:34.503250 19611 ts_tablet_manager.cc:1905] T 2b66f52ba6814a4d96f062c9c98a30c4 P 892d55ac7316407a9abe214a09505c40: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:11:34.503252 19609 ts_tablet_manager.cc:1905] T 2b66f52ba6814a4d96f062c9c98a30c4 P eb58cc0eb6a94359b8c1a4a3a8d9ba3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:11:34.506214 19611 ts_tablet_manager.cc:1918] T 2b66f52ba6814a4d96f062c9c98a30c4 P 892d55ac7316407a9abe214a09505c40: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:11:34.506320 19611 log.cc:1196] T 2b66f52ba6814a4d96f062c9c98a30c4 P 892d55ac7316407a9abe214a09505c40: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-1/wal/wals/2b66f52ba6814a4d96f062c9c98a30c4
I20260806 21:11:34.506642 19611 ts_tablet_manager.cc:1939] T 2b66f52ba6814a4d96f062c9c98a30c4 P 892d55ac7316407a9abe214a09505c40: Deleting consensus metadata
I20260806 21:11:34.507473 19609 ts_tablet_manager.cc:1918] T 2b66f52ba6814a4d96f062c9c98a30c4 P eb58cc0eb6a94359b8c1a4a3a8d9ba3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:11:34.507483 19418 tablet_service.cc:1467] Processing CreateTablet for tablet eee64a9c0af14646841676dcb0b762e6 (DEFAULT_TABLE table=test-table [id=6bdecb8b339346a18fbf9b32a283f5ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:11:34.507525 19225 tablet_service.cc:1467] Processing CreateTablet for tablet eee64a9c0af14646841676dcb0b762e6 (DEFAULT_TABLE table=test-table [id=6bdecb8b339346a18fbf9b32a283f5ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:11:34.507704 19609 log.cc:1196] T 2b66f52ba6814a4d96f062c9c98a30c4 P eb58cc0eb6a94359b8c1a4a3a8d9ba3a: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-0/wal/wals/2b66f52ba6814a4d96f062c9c98a30c4
I20260806 21:11:34.507776 19418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eee64a9c0af14646841676dcb0b762e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:34.507915 19002 tablet_service.cc:1467] Processing CreateTablet for tablet eee64a9c0af14646841676dcb0b762e6 (DEFAULT_TABLE table=test-table [id=6bdecb8b339346a18fbf9b32a283f5ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:11:34.508186 19609 ts_tablet_manager.cc:1939] T 2b66f52ba6814a4d96f062c9c98a30c4 P eb58cc0eb6a94359b8c1a4a3a8d9ba3a: Deleting consensus metadata
I20260806 21:11:34.508330 19225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eee64a9c0af14646841676dcb0b762e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:34.509662 18766 catalog_manager.cc:4841] TS eb58cc0eb6a94359b8c1a4a3a8d9ba3a (127.16.178.1:32823): tablet 2b66f52ba6814a4d96f062c9c98a30c4 (table test-table [id=3edced899a2e4960b6b27726ebf96a4e]) successfully deleted
I20260806 21:11:34.510402 19575 tablet_bootstrap.cc:492] T eee64a9c0af14646841676dcb0b762e6 P 5311aac7a92645e3967b76b74515ca20: Bootstrap starting.
I20260806 21:11:34.510448 19573 tablet_bootstrap.cc:492] T eee64a9c0af14646841676dcb0b762e6 P 892d55ac7316407a9abe214a09505c40: Bootstrap starting.
I20260806 21:11:34.508190 19002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eee64a9c0af14646841676dcb0b762e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:34.511482 19575 tablet_bootstrap.cc:654] T eee64a9c0af14646841676dcb0b762e6 P 5311aac7a92645e3967b76b74515ca20: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:34.511529 19573 tablet_bootstrap.cc:654] T eee64a9c0af14646841676dcb0b762e6 P 892d55ac7316407a9abe214a09505c40: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:34.511842 19608 tablet_replica.cc:330] T 2b66f52ba6814a4d96f062c9c98a30c4 P 5311aac7a92645e3967b76b74515ca20: stopping tablet replica
I20260806 21:11:34.512229 19608 raft_consensus.cc:2229] T 2b66f52ba6814a4d96f062c9c98a30c4 P 5311aac7a92645e3967b76b74515ca20 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:11:34.512616 19608 raft_consensus.cc:2258] T 2b66f52ba6814a4d96f062c9c98a30c4 P 5311aac7a92645e3967b76b74515ca20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:11:34.513679 19582 tablet_bootstrap.cc:492] T eee64a9c0af14646841676dcb0b762e6 P eb58cc0eb6a94359b8c1a4a3a8d9ba3a: Bootstrap starting.
I20260806 21:11:34.514204 18766 catalog_manager.cc:4841] TS 892d55ac7316407a9abe214a09505c40 (127.16.178.2:36933): tablet 2b66f52ba6814a4d96f062c9c98a30c4 (table test-table [id=3edced899a2e4960b6b27726ebf96a4e]) successfully deleted
I20260806 21:11:34.514734 19582 tablet_bootstrap.cc:654] T eee64a9c0af14646841676dcb0b762e6 P eb58cc0eb6a94359b8c1a4a3a8d9ba3a: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:34.514997 19608 ts_tablet_manager.cc:1905] T 2b66f52ba6814a4d96f062c9c98a30c4 P 5311aac7a92645e3967b76b74515ca20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:11:34.515168 19573 tablet_bootstrap.cc:492] T eee64a9c0af14646841676dcb0b762e6 P 892d55ac7316407a9abe214a09505c40: No bootstrap required, opened a new log
I20260806 21:11:34.515168 19575 tablet_bootstrap.cc:492] T eee64a9c0af14646841676dcb0b762e6 P 5311aac7a92645e3967b76b74515ca20: No bootstrap required, opened a new log
I20260806 21:11:34.515244 19575 ts_tablet_manager.cc:1397] T eee64a9c0af14646841676dcb0b762e6 P 5311aac7a92645e3967b76b74515ca20: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260806 21:11:34.515244 19573 ts_tablet_manager.cc:1397] T eee64a9c0af14646841676dcb0b762e6 P 892d55ac7316407a9abe214a09505c40: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260806 21:11:34.515705 19575 raft_consensus.cc:348] T eee64a9c0af14646841676dcb0b762e6 P 5311aac7a92645e3967b76b74515ca20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32823 } } peers { permanent_uuid: "892d55ac7316407a9abe214a09505c40" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 36933 } } peers { permanent_uuid: "5311aac7a92645e3967b76b74515ca20" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 38633 } }
I20260806 21:11:34.515705 19573 raft_consensus.cc:348] T eee64a9c0af14646841676dcb0b762e6 P 892d55ac7316407a9abe214a09505c40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32823 } } peers { permanent_uuid: "892d55ac7316407a9abe214a09505c40" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 36933 } } peers { permanent_uuid: "5311aac7a92645e3967b76b74515ca20" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 38633 } }
I20260806 21:11:34.515820 19575 raft_consensus.cc:374] T eee64a9c0af14646841676dcb0b762e6 P 5311aac7a92645e3967b76b74515ca20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:34.515827 19573 raft_consensus.cc:374] T eee64a9c0af14646841676dcb0b762e6 P 892d55ac7316407a9abe214a09505c40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:34.515853 19575 raft_consensus.cc:729] T eee64a9c0af14646841676dcb0b762e6 P 5311aac7a92645e3967b76b74515ca20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5311aac7a92645e3967b76b74515ca20, State: Initialized, Role: FOLLOWER
I20260806 21:11:34.515857 19573 raft_consensus.cc:729] T eee64a9c0af14646841676dcb0b762e6 P 892d55ac7316407a9abe214a09505c40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 892d55ac7316407a9abe214a09505c40, State: Initialized, Role: FOLLOWER
I20260806 21:11:34.515911 19582 tablet_bootstrap.cc:492] T eee64a9c0af14646841676dcb0b762e6 P eb58cc0eb6a94359b8c1a4a3a8d9ba3a: No bootstrap required, opened a new log
I20260806 21:11:34.516006 19582 ts_tablet_manager.cc:1397] T eee64a9c0af14646841676dcb0b762e6 P eb58cc0eb6a94359b8c1a4a3a8d9ba3a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260806 21:11:34.515973 19573 consensus_queue.cc:260] T eee64a9c0af14646841676dcb0b762e6 P 892d55ac7316407a9abe214a09505c40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32823 } } peers { permanent_uuid: "892d55ac7316407a9abe214a09505c40" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 36933 } } peers { permanent_uuid: "5311aac7a92645e3967b76b74515ca20" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 38633 } }
I20260806 21:11:34.515974 19575 consensus_queue.cc:260] T eee64a9c0af14646841676dcb0b762e6 P 5311aac7a92645e3967b76b74515ca20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32823 } } peers { permanent_uuid: "892d55ac7316407a9abe214a09505c40" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 36933 } } peers { permanent_uuid: "5311aac7a92645e3967b76b74515ca20" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 38633 } }
I20260806 21:11:34.516250 19573 ts_tablet_manager.cc:1428] T eee64a9c0af14646841676dcb0b762e6 P 892d55ac7316407a9abe214a09505c40: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 21:11:34.516629 19582 raft_consensus.cc:348] T eee64a9c0af14646841676dcb0b762e6 P eb58cc0eb6a94359b8c1a4a3a8d9ba3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32823 } } peers { permanent_uuid: "892d55ac7316407a9abe214a09505c40" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 36933 } } peers { permanent_uuid: "5311aac7a92645e3967b76b74515ca20" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 38633 } }
I20260806 21:11:34.516742 19582 raft_consensus.cc:374] T eee64a9c0af14646841676dcb0b762e6 P eb58cc0eb6a94359b8c1a4a3a8d9ba3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:34.516773 19582 raft_consensus.cc:729] T eee64a9c0af14646841676dcb0b762e6 P eb58cc0eb6a94359b8c1a4a3a8d9ba3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb58cc0eb6a94359b8c1a4a3a8d9ba3a, State: Initialized, Role: FOLLOWER
I20260806 21:11:34.516892 19582 consensus_queue.cc:260] T eee64a9c0af14646841676dcb0b762e6 P eb58cc0eb6a94359b8c1a4a3a8d9ba3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32823 } } peers { permanent_uuid: "892d55ac7316407a9abe214a09505c40" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 36933 } } peers { permanent_uuid: "5311aac7a92645e3967b76b74515ca20" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 38633 } }
I20260806 21:11:34.517146 19582 ts_tablet_manager.cc:1428] T eee64a9c0af14646841676dcb0b762e6 P eb58cc0eb6a94359b8c1a4a3a8d9ba3a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 21:11:34.517827 19608 ts_tablet_manager.cc:1918] T 2b66f52ba6814a4d96f062c9c98a30c4 P 5311aac7a92645e3967b76b74515ca20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:11:34.517928 19608 log.cc:1196] T 2b66f52ba6814a4d96f062c9c98a30c4 P 5311aac7a92645e3967b76b74515ca20: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-2/wal/wals/2b66f52ba6814a4d96f062c9c98a30c4
I20260806 21:11:34.518251 19608 ts_tablet_manager.cc:1939] T 2b66f52ba6814a4d96f062c9c98a30c4 P 5311aac7a92645e3967b76b74515ca20: Deleting consensus metadata
I20260806 21:11:34.519452 19575 ts_tablet_manager.cc:1428] T eee64a9c0af14646841676dcb0b762e6 P 5311aac7a92645e3967b76b74515ca20: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20260806 21:11:34.519614 18772 catalog_manager.cc:4841] TS 5311aac7a92645e3967b76b74515ca20 (127.16.178.3:38633): tablet 2b66f52ba6814a4d96f062c9c98a30c4 (table test-table [id=3edced899a2e4960b6b27726ebf96a4e]) successfully deleted
I20260806 21:11:34.703982 19603 raft_consensus.cc:482] T eee64a9c0af14646841676dcb0b762e6 P 892d55ac7316407a9abe214a09505c40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:11:34.704308 19603 raft_consensus.cc:504] T eee64a9c0af14646841676dcb0b762e6 P 892d55ac7316407a9abe214a09505c40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32823 } } peers { permanent_uuid: "892d55ac7316407a9abe214a09505c40" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 36933 } } peers { permanent_uuid: "5311aac7a92645e3967b76b74515ca20" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 38633 } }
I20260806 21:11:34.704813 19603 leader_election.cc:288] T eee64a9c0af14646841676dcb0b762e6 P 892d55ac7316407a9abe214a09505c40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb58cc0eb6a94359b8c1a4a3a8d9ba3a (127.16.178.1:32823), 5311aac7a92645e3967b76b74515ca20 (127.16.178.3:38633)
I20260806 21:11:34.705287 19039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eee64a9c0af14646841676dcb0b762e6" candidate_uuid: "892d55ac7316407a9abe214a09505c40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a" is_pre_election: true
I20260806 21:11:34.705294 19469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eee64a9c0af14646841676dcb0b762e6" candidate_uuid: "892d55ac7316407a9abe214a09505c40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5311aac7a92645e3967b76b74515ca20" is_pre_election: true
I20260806 21:11:34.705466 19469 raft_consensus.cc:2454] T eee64a9c0af14646841676dcb0b762e6 P 5311aac7a92645e3967b76b74515ca20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 892d55ac7316407a9abe214a09505c40 in term 0.
I20260806 21:11:34.705474 19039 raft_consensus.cc:2454] T eee64a9c0af14646841676dcb0b762e6 P eb58cc0eb6a94359b8c1a4a3a8d9ba3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 892d55ac7316407a9abe214a09505c40 in term 0.
I20260806 21:11:34.705794 19158 leader_election.cc:302] T eee64a9c0af14646841676dcb0b762e6 P 892d55ac7316407a9abe214a09505c40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5311aac7a92645e3967b76b74515ca20, 892d55ac7316407a9abe214a09505c40, eb58cc0eb6a94359b8c1a4a3a8d9ba3a; no voters:
I20260806 21:11:34.705983 19603 raft_consensus.cc:2781] T eee64a9c0af14646841676dcb0b762e6 P 892d55ac7316407a9abe214a09505c40 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:11:34.706044 19603 raft_consensus.cc:482] T eee64a9c0af14646841676dcb0b762e6 P 892d55ac7316407a9abe214a09505c40 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:11:34.706076 19603 raft_consensus.cc:3037] T eee64a9c0af14646841676dcb0b762e6 P 892d55ac7316407a9abe214a09505c40 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:34.707149 19603 raft_consensus.cc:504] T eee64a9c0af14646841676dcb0b762e6 P 892d55ac7316407a9abe214a09505c40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32823 } } peers { permanent_uuid: "892d55ac7316407a9abe214a09505c40" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 36933 } } peers { permanent_uuid: "5311aac7a92645e3967b76b74515ca20" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 38633 } }
I20260806 21:11:34.707541 19603 leader_election.cc:288] T eee64a9c0af14646841676dcb0b762e6 P 892d55ac7316407a9abe214a09505c40 [CANDIDATE]: Term 1 election: Requested vote from peers eb58cc0eb6a94359b8c1a4a3a8d9ba3a (127.16.178.1:32823), 5311aac7a92645e3967b76b74515ca20 (127.16.178.3:38633)
I20260806 21:11:34.707965 19039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eee64a9c0af14646841676dcb0b762e6" candidate_uuid: "892d55ac7316407a9abe214a09505c40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a"
I20260806 21:11:34.707968 19469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eee64a9c0af14646841676dcb0b762e6" candidate_uuid: "892d55ac7316407a9abe214a09505c40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5311aac7a92645e3967b76b74515ca20"
I20260806 21:11:34.708124 19469 raft_consensus.cc:3037] T eee64a9c0af14646841676dcb0b762e6 P 5311aac7a92645e3967b76b74515ca20 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:34.708125 19039 raft_consensus.cc:3037] T eee64a9c0af14646841676dcb0b762e6 P eb58cc0eb6a94359b8c1a4a3a8d9ba3a [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:34.709076 19039 raft_consensus.cc:2454] T eee64a9c0af14646841676dcb0b762e6 P eb58cc0eb6a94359b8c1a4a3a8d9ba3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 892d55ac7316407a9abe214a09505c40 in term 1.
I20260806 21:11:34.709088 19469 raft_consensus.cc:2454] T eee64a9c0af14646841676dcb0b762e6 P 5311aac7a92645e3967b76b74515ca20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 892d55ac7316407a9abe214a09505c40 in term 1.
I20260806 21:11:34.709396 19155 leader_election.cc:302] T eee64a9c0af14646841676dcb0b762e6 P 892d55ac7316407a9abe214a09505c40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 892d55ac7316407a9abe214a09505c40, eb58cc0eb6a94359b8c1a4a3a8d9ba3a; no voters:
I20260806 21:11:34.709599 19603 raft_consensus.cc:2781] T eee64a9c0af14646841676dcb0b762e6 P 892d55ac7316407a9abe214a09505c40 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:11:34.709689 19603 raft_consensus.cc:686] T eee64a9c0af14646841676dcb0b762e6 P 892d55ac7316407a9abe214a09505c40 [term 1 LEADER]: Becoming Leader. State: Replica: 892d55ac7316407a9abe214a09505c40, State: Running, Role: LEADER
I20260806 21:11:34.709841 19603 consensus_queue.cc:237] T eee64a9c0af14646841676dcb0b762e6 P 892d55ac7316407a9abe214a09505c40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32823 } } peers { permanent_uuid: "892d55ac7316407a9abe214a09505c40" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 36933 } } peers { permanent_uuid: "5311aac7a92645e3967b76b74515ca20" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 38633 } }
I20260806 21:11:34.711589 18789 catalog_manager.cc:5495] T eee64a9c0af14646841676dcb0b762e6 P 892d55ac7316407a9abe214a09505c40 reported cstate change: term changed from 0 to 1, leader changed from <none> to 892d55ac7316407a9abe214a09505c40 (127.16.178.2). New cstate: current_term: 1 leader_uuid: "892d55ac7316407a9abe214a09505c40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "892d55ac7316407a9abe214a09505c40" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 36933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5311aac7a92645e3967b76b74515ca20" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 38633 } health_report { overall_health: UNKNOWN } } }
Aug 06 21:11:34 dist-test-slave-lgr5 krb5kdc[18717](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050693, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:11:34.790374 19626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:34.782058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:39949 (local address 127.0.0.1:53236)
0806 21:11:34.782457 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:34.782470 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:34.782592 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:34.782841 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:34.782843 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:34.783276 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:34.783389 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:34.783397 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:34.784182 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:34.784186 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:34.784775 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:34.784784 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:34.785220 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:34.785223 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:34.785297 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:34.785926 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:34.785939 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:34.787423 (+ 1484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:34.788932 (+ 1509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:34.788942 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:34.788948 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:34.789178 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:34.789395 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:34.789397 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:34.789398 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:34.789471 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:34.789840 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:34.789860 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:34.789985 (+ 125us) client_negotiation.cc:769] Sending connection context
0806 21:11:34.790089 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 21:11:34.790221 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":111,"threads_started":1}
I20260806 21:11:34.790408 19627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:34.782110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53236 (local address 127.16.178.62:39949)
0806 21:11:34.782384 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:34.782393 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:34.782541 (+ 148us) server_negotiation.cc:407] Connection header received
0806 21:11:34.782692 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:34.782695 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:34.782737 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:34.782809 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:34.783510 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:34.784061 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:34.784886 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:34.785115 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:34.785201 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:34.787581 (+ 2380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:34.787584 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:34.787586 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:34.787610 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:34.788828 (+ 1218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:34.789258 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:34.789260 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:34.789262 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:34.789331 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:34.789532 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:34.789534 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:34.789535 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:34.789709 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:34.789781 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:34.790134 (+ 353us) server_negotiation.cc:299] Negotiation successful
0806 21:11:34.790283 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":187,"thread_start_us":93,"threads_started":1}
I20260806 21:11:34.794757 18789 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:53236:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 21:11:34.795545 18789 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:11:34.800457 19225 tablet_service.cc:1467] Processing CreateTablet for tablet 437aaee8aa4547df9ae564797c9dea5c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bd812e28afe5461b8ce76f000a63d89a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 21:11:34.800532 19418 tablet_service.cc:1467] Processing CreateTablet for tablet 437aaee8aa4547df9ae564797c9dea5c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bd812e28afe5461b8ce76f000a63d89a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 21:11:34.800460 19002 tablet_service.cc:1467] Processing CreateTablet for tablet 437aaee8aa4547df9ae564797c9dea5c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bd812e28afe5461b8ce76f000a63d89a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 21:11:34.800940 19418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 437aaee8aa4547df9ae564797c9dea5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:34.800949 19002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 437aaee8aa4547df9ae564797c9dea5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:34.800944 19225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 437aaee8aa4547df9ae564797c9dea5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:34.803452 19573 tablet_bootstrap.cc:492] T 437aaee8aa4547df9ae564797c9dea5c P 892d55ac7316407a9abe214a09505c40: Bootstrap starting.
I20260806 21:11:34.803798 19582 tablet_bootstrap.cc:492] T 437aaee8aa4547df9ae564797c9dea5c P eb58cc0eb6a94359b8c1a4a3a8d9ba3a: Bootstrap starting.
I20260806 21:11:34.804066 19575 tablet_bootstrap.cc:492] T 437aaee8aa4547df9ae564797c9dea5c P 5311aac7a92645e3967b76b74515ca20: Bootstrap starting.
I20260806 21:11:34.804388 19573 tablet_bootstrap.cc:654] T 437aaee8aa4547df9ae564797c9dea5c P 892d55ac7316407a9abe214a09505c40: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:34.804767 19582 tablet_bootstrap.cc:654] T 437aaee8aa4547df9ae564797c9dea5c P eb58cc0eb6a94359b8c1a4a3a8d9ba3a: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:34.805032 19575 tablet_bootstrap.cc:654] T 437aaee8aa4547df9ae564797c9dea5c P 5311aac7a92645e3967b76b74515ca20: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:34.805627 19573 tablet_bootstrap.cc:492] T 437aaee8aa4547df9ae564797c9dea5c P 892d55ac7316407a9abe214a09505c40: No bootstrap required, opened a new log
I20260806 21:11:34.805716 19573 ts_tablet_manager.cc:1397] T 437aaee8aa4547df9ae564797c9dea5c P 892d55ac7316407a9abe214a09505c40: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260806 21:11:34.806191 19575 tablet_bootstrap.cc:492] T 437aaee8aa4547df9ae564797c9dea5c P 5311aac7a92645e3967b76b74515ca20: No bootstrap required, opened a new log
I20260806 21:11:34.806137 19573 raft_consensus.cc:348] T 437aaee8aa4547df9ae564797c9dea5c P 892d55ac7316407a9abe214a09505c40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32823 } } peers { permanent_uuid: "892d55ac7316407a9abe214a09505c40" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 36933 } } peers { permanent_uuid: "5311aac7a92645e3967b76b74515ca20" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 38633 } }
I20260806 21:11:34.806241 19582 tablet_bootstrap.cc:492] T 437aaee8aa4547df9ae564797c9dea5c P eb58cc0eb6a94359b8c1a4a3a8d9ba3a: No bootstrap required, opened a new log
I20260806 21:11:34.806254 19573 raft_consensus.cc:374] T 437aaee8aa4547df9ae564797c9dea5c P 892d55ac7316407a9abe214a09505c40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:34.806263 19575 ts_tablet_manager.cc:1397] T 437aaee8aa4547df9ae564797c9dea5c P 5311aac7a92645e3967b76b74515ca20: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 21:11:34.806288 19573 raft_consensus.cc:729] T 437aaee8aa4547df9ae564797c9dea5c P 892d55ac7316407a9abe214a09505c40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 892d55ac7316407a9abe214a09505c40, State: Initialized, Role: FOLLOWER
I20260806 21:11:34.806301 19582 ts_tablet_manager.cc:1397] T 437aaee8aa4547df9ae564797c9dea5c P eb58cc0eb6a94359b8c1a4a3a8d9ba3a: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 21:11:34.806424 19573 consensus_queue.cc:260] T 437aaee8aa4547df9ae564797c9dea5c P 892d55ac7316407a9abe214a09505c40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32823 } } peers { permanent_uuid: "892d55ac7316407a9abe214a09505c40" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 36933 } } peers { permanent_uuid: "5311aac7a92645e3967b76b74515ca20" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 38633 } }
I20260806 21:11:34.806720 19573 ts_tablet_manager.cc:1428] T 437aaee8aa4547df9ae564797c9dea5c P 892d55ac7316407a9abe214a09505c40: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 21:11:34.806692 19575 raft_consensus.cc:348] T 437aaee8aa4547df9ae564797c9dea5c P 5311aac7a92645e3967b76b74515ca20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32823 } } peers { permanent_uuid: "892d55ac7316407a9abe214a09505c40" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 36933 } } peers { permanent_uuid: "5311aac7a92645e3967b76b74515ca20" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 38633 } }
I20260806 21:11:34.806711 19582 raft_consensus.cc:348] T 437aaee8aa4547df9ae564797c9dea5c P eb58cc0eb6a94359b8c1a4a3a8d9ba3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32823 } } peers { permanent_uuid: "892d55ac7316407a9abe214a09505c40" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 36933 } } peers { permanent_uuid: "5311aac7a92645e3967b76b74515ca20" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 38633 } }
I20260806 21:11:34.806794 19575 raft_consensus.cc:374] T 437aaee8aa4547df9ae564797c9dea5c P 5311aac7a92645e3967b76b74515ca20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:34.806825 19575 raft_consensus.cc:729] T 437aaee8aa4547df9ae564797c9dea5c P 5311aac7a92645e3967b76b74515ca20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5311aac7a92645e3967b76b74515ca20, State: Initialized, Role: FOLLOWER
I20260806 21:11:34.806828 19582 raft_consensus.cc:374] T 437aaee8aa4547df9ae564797c9dea5c P eb58cc0eb6a94359b8c1a4a3a8d9ba3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:34.806859 19582 raft_consensus.cc:729] T 437aaee8aa4547df9ae564797c9dea5c P eb58cc0eb6a94359b8c1a4a3a8d9ba3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb58cc0eb6a94359b8c1a4a3a8d9ba3a, State: Initialized, Role: FOLLOWER
I20260806 21:11:34.806792 19603 tablet_replica.cc:439] T 437aaee8aa4547df9ae564797c9dea5c P 892d55ac7316407a9abe214a09505c40: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32823 } } peers { permanent_uuid: "892d55ac7316407a9abe214a09505c40" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 36933 } } peers { permanent_uuid: "5311aac7a92645e3967b76b74515ca20" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 38633 } } }
I20260806 21:11:34.806929 19603 tablet_replica.cc:442] T 437aaee8aa4547df9ae564797c9dea5c P 892d55ac7316407a9abe214a09505c40: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 21:11:34.806938 19575 consensus_queue.cc:260] T 437aaee8aa4547df9ae564797c9dea5c P 5311aac7a92645e3967b76b74515ca20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32823 } } peers { permanent_uuid: "892d55ac7316407a9abe214a09505c40" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 36933 } } peers { permanent_uuid: "5311aac7a92645e3967b76b74515ca20" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 38633 } }
I20260806 21:11:34.806993 19582 consensus_queue.cc:260] T 437aaee8aa4547df9ae564797c9dea5c P eb58cc0eb6a94359b8c1a4a3a8d9ba3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32823 } } peers { permanent_uuid: "892d55ac7316407a9abe214a09505c40" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 36933 } } peers { permanent_uuid: "5311aac7a92645e3967b76b74515ca20" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 38633 } }
I20260806 21:11:34.807214 19575 ts_tablet_manager.cc:1428] T 437aaee8aa4547df9ae564797c9dea5c P 5311aac7a92645e3967b76b74515ca20: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 21:11:34.807253 19582 ts_tablet_manager.cc:1428] T 437aaee8aa4547df9ae564797c9dea5c P eb58cc0eb6a94359b8c1a4a3a8d9ba3a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 21:11:34.807323 19586 tablet_replica.cc:439] T 437aaee8aa4547df9ae564797c9dea5c P 5311aac7a92645e3967b76b74515ca20: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32823 } } peers { permanent_uuid: "892d55ac7316407a9abe214a09505c40" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 36933 } } peers { permanent_uuid: "5311aac7a92645e3967b76b74515ca20" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 38633 } } }
I20260806 21:11:34.807480 19586 tablet_replica.cc:442] T 437aaee8aa4547df9ae564797c9dea5c P 5311aac7a92645e3967b76b74515ca20: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 21:11:34.807453 19588 tablet_replica.cc:439] T 437aaee8aa4547df9ae564797c9dea5c P eb58cc0eb6a94359b8c1a4a3a8d9ba3a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32823 } } peers { permanent_uuid: "892d55ac7316407a9abe214a09505c40" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 36933 } } peers { permanent_uuid: "5311aac7a92645e3967b76b74515ca20" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 38633 } } }
I20260806 21:11:34.807569 19588 tablet_replica.cc:442] T 437aaee8aa4547df9ae564797c9dea5c P eb58cc0eb6a94359b8c1a4a3a8d9ba3a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 21:11:34.874828 19603 raft_consensus.cc:482] T 437aaee8aa4547df9ae564797c9dea5c P 892d55ac7316407a9abe214a09505c40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:11:34.874985 19603 raft_consensus.cc:504] T 437aaee8aa4547df9ae564797c9dea5c P 892d55ac7316407a9abe214a09505c40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32823 } } peers { permanent_uuid: "892d55ac7316407a9abe214a09505c40" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 36933 } } peers { permanent_uuid: "5311aac7a92645e3967b76b74515ca20" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 38633 } }
I20260806 21:11:34.875488 19603 leader_election.cc:288] T 437aaee8aa4547df9ae564797c9dea5c P 892d55ac7316407a9abe214a09505c40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb58cc0eb6a94359b8c1a4a3a8d9ba3a (127.16.178.1:32823), 5311aac7a92645e3967b76b74515ca20 (127.16.178.3:38633)
I20260806 21:11:34.875980 19039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "437aaee8aa4547df9ae564797c9dea5c" candidate_uuid: "892d55ac7316407a9abe214a09505c40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a" is_pre_election: true
I20260806 21:11:34.875980 19469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "437aaee8aa4547df9ae564797c9dea5c" candidate_uuid: "892d55ac7316407a9abe214a09505c40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5311aac7a92645e3967b76b74515ca20" is_pre_election: true
I20260806 21:11:34.876154 19039 raft_consensus.cc:2454] T 437aaee8aa4547df9ae564797c9dea5c P eb58cc0eb6a94359b8c1a4a3a8d9ba3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 892d55ac7316407a9abe214a09505c40 in term 0.
I20260806 21:11:34.876165 19469 raft_consensus.cc:2454] T 437aaee8aa4547df9ae564797c9dea5c P 5311aac7a92645e3967b76b74515ca20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 892d55ac7316407a9abe214a09505c40 in term 0.
I20260806 21:11:34.876453 19155 leader_election.cc:302] T 437aaee8aa4547df9ae564797c9dea5c P 892d55ac7316407a9abe214a09505c40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 892d55ac7316407a9abe214a09505c40, eb58cc0eb6a94359b8c1a4a3a8d9ba3a; no voters:
I20260806 21:11:34.876629 19603 raft_consensus.cc:2781] T 437aaee8aa4547df9ae564797c9dea5c P 892d55ac7316407a9abe214a09505c40 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:11:34.876693 19603 raft_consensus.cc:482] T 437aaee8aa4547df9ae564797c9dea5c P 892d55ac7316407a9abe214a09505c40 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:11:34.876720 19603 raft_consensus.cc:3037] T 437aaee8aa4547df9ae564797c9dea5c P 892d55ac7316407a9abe214a09505c40 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:34.877677 19603 raft_consensus.cc:504] T 437aaee8aa4547df9ae564797c9dea5c P 892d55ac7316407a9abe214a09505c40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32823 } } peers { permanent_uuid: "892d55ac7316407a9abe214a09505c40" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 36933 } } peers { permanent_uuid: "5311aac7a92645e3967b76b74515ca20" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 38633 } }
I20260806 21:11:34.877983 19603 leader_election.cc:288] T 437aaee8aa4547df9ae564797c9dea5c P 892d55ac7316407a9abe214a09505c40 [CANDIDATE]: Term 1 election: Requested vote from peers eb58cc0eb6a94359b8c1a4a3a8d9ba3a (127.16.178.1:32823), 5311aac7a92645e3967b76b74515ca20 (127.16.178.3:38633)
I20260806 21:11:34.878331 19039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "437aaee8aa4547df9ae564797c9dea5c" candidate_uuid: "892d55ac7316407a9abe214a09505c40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a"
I20260806 21:11:34.878350 19469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "437aaee8aa4547df9ae564797c9dea5c" candidate_uuid: "892d55ac7316407a9abe214a09505c40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5311aac7a92645e3967b76b74515ca20"
I20260806 21:11:34.878449 19039 raft_consensus.cc:3037] T 437aaee8aa4547df9ae564797c9dea5c P eb58cc0eb6a94359b8c1a4a3a8d9ba3a [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:34.878446 19469 raft_consensus.cc:3037] T 437aaee8aa4547df9ae564797c9dea5c P 5311aac7a92645e3967b76b74515ca20 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:34.879338 19039 raft_consensus.cc:2454] T 437aaee8aa4547df9ae564797c9dea5c P eb58cc0eb6a94359b8c1a4a3a8d9ba3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 892d55ac7316407a9abe214a09505c40 in term 1.
I20260806 21:11:34.879398 19469 raft_consensus.cc:2454] T 437aaee8aa4547df9ae564797c9dea5c P 5311aac7a92645e3967b76b74515ca20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 892d55ac7316407a9abe214a09505c40 in term 1.
I20260806 21:11:34.879614 19155 leader_election.cc:302] T 437aaee8aa4547df9ae564797c9dea5c P 892d55ac7316407a9abe214a09505c40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 892d55ac7316407a9abe214a09505c40, eb58cc0eb6a94359b8c1a4a3a8d9ba3a; no voters:
I20260806 21:11:34.879825 19603 raft_consensus.cc:2781] T 437aaee8aa4547df9ae564797c9dea5c P 892d55ac7316407a9abe214a09505c40 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:11:34.879913 19603 raft_consensus.cc:686] T 437aaee8aa4547df9ae564797c9dea5c P 892d55ac7316407a9abe214a09505c40 [term 1 LEADER]: Becoming Leader. State: Replica: 892d55ac7316407a9abe214a09505c40, State: Running, Role: LEADER
I20260806 21:11:34.880059 19603 consensus_queue.cc:237] T 437aaee8aa4547df9ae564797c9dea5c P 892d55ac7316407a9abe214a09505c40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32823 } } peers { permanent_uuid: "892d55ac7316407a9abe214a09505c40" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 36933 } } peers { permanent_uuid: "5311aac7a92645e3967b76b74515ca20" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 38633 } }
I20260806 21:11:34.880751 19583 tablet_replica.cc:439] T 437aaee8aa4547df9ae564797c9dea5c P 892d55ac7316407a9abe214a09505c40: TxnStatusTablet state changed. Reason: New leader 892d55ac7316407a9abe214a09505c40. Latest consensus state: current_term: 1 leader_uuid: "892d55ac7316407a9abe214a09505c40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32823 } } peers { permanent_uuid: "892d55ac7316407a9abe214a09505c40" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 36933 } } peers { permanent_uuid: "5311aac7a92645e3967b76b74515ca20" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 38633 } } }
I20260806 21:11:34.880857 19583 tablet_replica.cc:442] T 437aaee8aa4547df9ae564797c9dea5c P 892d55ac7316407a9abe214a09505c40: This TxnStatusTablet replica's current role is: LEADER
I20260806 21:11:34.881210 19633 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 21:11:34.881425 19633 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 21:11:34.881870 18793 catalog_manager.cc:5495] T 437aaee8aa4547df9ae564797c9dea5c P 892d55ac7316407a9abe214a09505c40 reported cstate change: term changed from 0 to 1, leader changed from <none> to 892d55ac7316407a9abe214a09505c40 (127.16.178.2). New cstate: current_term: 1 leader_uuid: "892d55ac7316407a9abe214a09505c40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "892d55ac7316407a9abe214a09505c40" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 36933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5311aac7a92645e3967b76b74515ca20" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 38633 } health_report { overall_health: UNKNOWN } } }
I20260806 21:11:34.895064 19572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:34.891146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56282 (local address 127.16.178.2:36933)
0806 21:11:34.891268 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:34.891272 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:34.891375 (+ 103us) server_negotiation.cc:407] Connection header received
0806 21:11:34.891544 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:34.891547 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:34.891593 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:34.891656 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:11:34.891887 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:34.892532 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:34.893867 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:34.894127 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:34.894215 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:34.894603 (+ 388us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:11:34.894690 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:11:34.894748 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:34.894852 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 21:11:34.894916 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 21:11:34.895133 19626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:34.891058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.2:36933 (local address 127.0.0.1:56282)
0806 21:11:34.891304 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:34.891318 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:34.891437 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:34.891684 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:34.891687 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:34.891694 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 21:11:34.891807 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:34.891813 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:34.892648 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:34.892652 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:34.893694 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:34.893707 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:34.894261 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:34.894265 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:34.894366 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:34.894444 (+ 78us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 21:11:34.894764 (+ 320us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 21:11:34.894777 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 21:11:34.894839 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 21:11:34.894953 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142}
I20260806 21:11:34.898077 19469 raft_consensus.cc:1261] T 437aaee8aa4547df9ae564797c9dea5c P 5311aac7a92645e3967b76b74515ca20 [term 1 FOLLOWER]: Refusing update from remote peer 892d55ac7316407a9abe214a09505c40: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:11:34.898078 19039 raft_consensus.cc:1261] T 437aaee8aa4547df9ae564797c9dea5c P eb58cc0eb6a94359b8c1a4a3a8d9ba3a [term 1 FOLLOWER]: Refusing update from remote peer 892d55ac7316407a9abe214a09505c40: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:11:34.898494 19583 consensus_queue.cc:1035] T 437aaee8aa4547df9ae564797c9dea5c P 892d55ac7316407a9abe214a09505c40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5311aac7a92645e3967b76b74515ca20" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 38633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:11:34.898660 19603 consensus_queue.cc:1035] T 437aaee8aa4547df9ae564797c9dea5c P 892d55ac7316407a9abe214a09505c40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:11:34.900873 19586 tablet_replica.cc:439] T 437aaee8aa4547df9ae564797c9dea5c P 5311aac7a92645e3967b76b74515ca20: TxnStatusTablet state changed. Reason: New leader 892d55ac7316407a9abe214a09505c40. Latest consensus state: current_term: 1 leader_uuid: "892d55ac7316407a9abe214a09505c40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32823 } } peers { permanent_uuid: "892d55ac7316407a9abe214a09505c40" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 36933 } } peers { permanent_uuid: "5311aac7a92645e3967b76b74515ca20" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 38633 } } }
I20260806 21:11:34.901010 19586 tablet_replica.cc:442] T 437aaee8aa4547df9ae564797c9dea5c P 5311aac7a92645e3967b76b74515ca20: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 21:11:34.902180 19588 tablet_replica.cc:439] T 437aaee8aa4547df9ae564797c9dea5c P eb58cc0eb6a94359b8c1a4a3a8d9ba3a: TxnStatusTablet state changed. Reason: New leader 892d55ac7316407a9abe214a09505c40. Latest consensus state: current_term: 1 leader_uuid: "892d55ac7316407a9abe214a09505c40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32823 } } peers { permanent_uuid: "892d55ac7316407a9abe214a09505c40" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 36933 } } peers { permanent_uuid: "5311aac7a92645e3967b76b74515ca20" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 38633 } } }
I20260806 21:11:34.902300 19588 tablet_replica.cc:442] T 437aaee8aa4547df9ae564797c9dea5c P eb58cc0eb6a94359b8c1a4a3a8d9ba3a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 21:11:34.903201 19583 tablet_replica.cc:439] T 437aaee8aa4547df9ae564797c9dea5c P 892d55ac7316407a9abe214a09505c40: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "892d55ac7316407a9abe214a09505c40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32823 } } peers { permanent_uuid: "892d55ac7316407a9abe214a09505c40" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 36933 } } peers { permanent_uuid: "5311aac7a92645e3967b76b74515ca20" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 38633 } } }
I20260806 21:11:34.903322 19583 tablet_replica.cc:442] T 437aaee8aa4547df9ae564797c9dea5c P 892d55ac7316407a9abe214a09505c40: This TxnStatusTablet replica's current role is: LEADER
I20260806 21:11:34.903784 19583 tablet_replica.cc:439] T 437aaee8aa4547df9ae564797c9dea5c P 892d55ac7316407a9abe214a09505c40: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "892d55ac7316407a9abe214a09505c40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32823 } } peers { permanent_uuid: "892d55ac7316407a9abe214a09505c40" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 36933 } } peers { permanent_uuid: "5311aac7a92645e3967b76b74515ca20" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 38633 } } }
I20260806 21:11:34.903878 19583 tablet_replica.cc:442] T 437aaee8aa4547df9ae564797c9dea5c P 892d55ac7316407a9abe214a09505c40: This TxnStatusTablet replica's current role is: LEADER
I20260806 21:11:34.904759 19586 tablet_replica.cc:439] T 437aaee8aa4547df9ae564797c9dea5c P 5311aac7a92645e3967b76b74515ca20: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "892d55ac7316407a9abe214a09505c40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32823 } } peers { permanent_uuid: "892d55ac7316407a9abe214a09505c40" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 36933 } } peers { permanent_uuid: "5311aac7a92645e3967b76b74515ca20" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 38633 } } }
I20260806 21:11:34.904868 19586 tablet_replica.cc:442] T 437aaee8aa4547df9ae564797c9dea5c P 5311aac7a92645e3967b76b74515ca20: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 21:11:34.905280 19596 mvcc.cc:205] Tried to move back new op lower bound from 7315663646298091520 to 7315663646229999616. Current Snapshot: MvccSnapshot[applied={T|T < 7315663646298091520 or (T in {7315663646298091520})}]
I20260806 21:11:34.906144 19588 tablet_replica.cc:439] T 437aaee8aa4547df9ae564797c9dea5c P eb58cc0eb6a94359b8c1a4a3a8d9ba3a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "892d55ac7316407a9abe214a09505c40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32823 } } peers { permanent_uuid: "892d55ac7316407a9abe214a09505c40" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 36933 } } peers { permanent_uuid: "5311aac7a92645e3967b76b74515ca20" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 38633 } } }
I20260806 21:11:34.906255 19588 tablet_replica.cc:442] T 437aaee8aa4547df9ae564797c9dea5c P eb58cc0eb6a94359b8c1a4a3a8d9ba3a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 21:11:34.922168 19572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:34.917362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:39861 (local address 127.16.178.2:36933)
0806 21:11:34.917491 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:34.917496 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:34.917749 (+ 253us) server_negotiation.cc:407] Connection header received
0806 21:11:34.917902 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:34.917905 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:34.917958 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:34.918031 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:11:34.918317 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:34.919079 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:34.920299 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:34.920670 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:34.920764 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:34.921628 (+ 864us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:11:34.921715 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:11:34.921782 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:34.921947 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 21:11:34.922012 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":51}
I20260806 21:11:34.922181 19646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:34.917297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.2:36933 (local address 127.16.178.2:39861)
0806 21:11:34.917656 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:34.917671 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:34.917789 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:34.918044 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:34.918047 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:34.918055 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 21:11:34.918168 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:34.918174 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:34.919184 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:34.919187 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:34.920187 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:34.920198 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:34.921342 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:34.921345 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:34.921436 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:34.921490 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 21:11:34.921846 (+ 356us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 21:11:34.921853 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 21:11:34.921929 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 21:11:34.922020 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":121,"threads_started":1}
I20260806 21:11:34.930310 19039 raft_consensus.cc:1261] T eee64a9c0af14646841676dcb0b762e6 P eb58cc0eb6a94359b8c1a4a3a8d9ba3a [term 1 FOLLOWER]: Refusing update from remote peer 892d55ac7316407a9abe214a09505c40: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:11:34.930374 19469 raft_consensus.cc:1261] T eee64a9c0af14646841676dcb0b762e6 P 5311aac7a92645e3967b76b74515ca20 [term 1 FOLLOWER]: Refusing update from remote peer 892d55ac7316407a9abe214a09505c40: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:11:34.930786 19592 consensus_queue.cc:1035] T eee64a9c0af14646841676dcb0b762e6 P 892d55ac7316407a9abe214a09505c40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb58cc0eb6a94359b8c1a4a3a8d9ba3a" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:11:34.931192 19592 consensus_queue.cc:1035] T eee64a9c0af14646841676dcb0b762e6 P 892d55ac7316407a9abe214a09505c40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5311aac7a92645e3967b76b74515ca20" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 38633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:11:34.979023 18789 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53222:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:11:34.979195 18789 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53222:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:11:34.981492 18789 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bdbe3f02fb6e499ca8be9c1521c8f20f: Sending DeleteTablet for 3 replicas of tablet eee64a9c0af14646841676dcb0b762e6
I20260806 21:11:34.982245 19225 tablet_service.cc:1514] Processing DeleteTablet for tablet eee64a9c0af14646841676dcb0b762e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:11:34 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:56274
I20260806 21:11:34.982267 19418 tablet_service.cc:1514] Processing DeleteTablet for tablet eee64a9c0af14646841676dcb0b762e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:11:34 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:55054
I20260806 21:11:34.982481 19611 tablet_replica.cc:330] T eee64a9c0af14646841676dcb0b762e6 P 892d55ac7316407a9abe214a09505c40: stopping tablet replica
I20260806 21:11:34.982483 19002 tablet_service.cc:1514] Processing DeleteTablet for tablet eee64a9c0af14646841676dcb0b762e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:11:34 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:43620
I20260806 21:11:34.982625 19611 raft_consensus.cc:2229] T eee64a9c0af14646841676dcb0b762e6 P 892d55ac7316407a9abe214a09505c40 [term 1 LEADER]: Raft consensus shutting down.
I20260806 21:11:34.982673 19608 tablet_replica.cc:330] T eee64a9c0af14646841676dcb0b762e6 P 5311aac7a92645e3967b76b74515ca20: stopping tablet replica
I20260806 21:11:34.982669 19609 tablet_replica.cc:330] T eee64a9c0af14646841676dcb0b762e6 P eb58cc0eb6a94359b8c1a4a3a8d9ba3a: stopping tablet replica
I20260806 21:11:34.982779 19608 raft_consensus.cc:2229] T eee64a9c0af14646841676dcb0b762e6 P 5311aac7a92645e3967b76b74515ca20 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:11:34.982806 19609 raft_consensus.cc:2229] T eee64a9c0af14646841676dcb0b762e6 P eb58cc0eb6a94359b8c1a4a3a8d9ba3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:11:34.982846 19611 raft_consensus.cc:2258] T eee64a9c0af14646841676dcb0b762e6 P 892d55ac7316407a9abe214a09505c40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:11:34.982892 19608 raft_consensus.cc:2258] T eee64a9c0af14646841676dcb0b762e6 P 5311aac7a92645e3967b76b74515ca20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:11:34.982932 19609 raft_consensus.cc:2258] T eee64a9c0af14646841676dcb0b762e6 P eb58cc0eb6a94359b8c1a4a3a8d9ba3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:11:34.983772 19608 ts_tablet_manager.cc:1905] T eee64a9c0af14646841676dcb0b762e6 P 5311aac7a92645e3967b76b74515ca20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:11:34.983771 19611 ts_tablet_manager.cc:1905] T eee64a9c0af14646841676dcb0b762e6 P 892d55ac7316407a9abe214a09505c40: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:11:34.983809 19609 ts_tablet_manager.cc:1905] T eee64a9c0af14646841676dcb0b762e6 P eb58cc0eb6a94359b8c1a4a3a8d9ba3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:11:34.986698 19611 ts_tablet_manager.cc:1918] T eee64a9c0af14646841676dcb0b762e6 P 892d55ac7316407a9abe214a09505c40: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 21:11:34.986773 19611 log.cc:1196] T eee64a9c0af14646841676dcb0b762e6 P 892d55ac7316407a9abe214a09505c40: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-1/wal/wals/eee64a9c0af14646841676dcb0b762e6
I20260806 21:11:34.986948 19611 ts_tablet_manager.cc:1939] T eee64a9c0af14646841676dcb0b762e6 P 892d55ac7316407a9abe214a09505c40: Deleting consensus metadata
I20260806 21:11:34.986974 19608 ts_tablet_manager.cc:1918] T eee64a9c0af14646841676dcb0b762e6 P 5311aac7a92645e3967b76b74515ca20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 21:11:34.987056 19608 log.cc:1196] T eee64a9c0af14646841676dcb0b762e6 P 5311aac7a92645e3967b76b74515ca20: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-2/wal/wals/eee64a9c0af14646841676dcb0b762e6
I20260806 21:11:34.987293 19608 ts_tablet_manager.cc:1939] T eee64a9c0af14646841676dcb0b762e6 P 5311aac7a92645e3967b76b74515ca20: Deleting consensus metadata
I20260806 21:11:34.988191 18766 catalog_manager.cc:4841] TS 892d55ac7316407a9abe214a09505c40 (127.16.178.2:36933): tablet eee64a9c0af14646841676dcb0b762e6 (table test-table [id=6bdecb8b339346a18fbf9b32a283f5ee]) successfully deleted
I20260806 21:11:34.989558 19609 ts_tablet_manager.cc:1918] T eee64a9c0af14646841676dcb0b762e6 P eb58cc0eb6a94359b8c1a4a3a8d9ba3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 21:11:34.989656 19609 log.cc:1196] T eee64a9c0af14646841676dcb0b762e6 P eb58cc0eb6a94359b8c1a4a3a8d9ba3a: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786050686797304-17096-0/minicluster-data/ts-0/wal/wals/eee64a9c0af14646841676dcb0b762e6
I20260806 21:11:34.989900 19609 ts_tablet_manager.cc:1939] T eee64a9c0af14646841676dcb0b762e6 P eb58cc0eb6a94359b8c1a4a3a8d9ba3a: Deleting consensus metadata
I20260806 21:11:34.990742 18772 catalog_manager.cc:4841] TS 5311aac7a92645e3967b76b74515ca20 (127.16.178.3:38633): tablet eee64a9c0af14646841676dcb0b762e6 (table test-table [id=6bdecb8b339346a18fbf9b32a283f5ee]) successfully deleted
I20260806 21:11:34.990737 18766 catalog_manager.cc:4841] TS eb58cc0eb6a94359b8c1a4a3a8d9ba3a (127.16.178.1:32823): tablet eee64a9c0af14646841676dcb0b762e6 (table test-table [id=6bdecb8b339346a18fbf9b32a283f5ee]) successfully deleted
Aug 06 21:11:34 dist-test-slave-lgr5 krb5kdc[18717](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050694, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.178.1@KRBTEST.COM
I20260806 21:11:35.000792 19652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:34.989630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43632 (local address 127.16.178.1:32823)
0806 21:11:34.989891 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:34.989895 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:34.990232 (+ 337us) server_negotiation.cc:407] Connection header received
0806 21:11:34.990365 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:34.990368 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:34.990420 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:34.990521 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:34.991551 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:34.992443 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:34.993412 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:34.993726 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:34.993838 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:34.996777 (+ 2939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:34.996796 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:34.996819 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:34.996855 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:34.998836 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:34.999322 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:34.999329 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:34.999331 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:34.999432 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:34.999668 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:34.999671 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:34.999672 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:35.000030 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:35.000312 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:35.000453 (+ 141us) server_negotiation.cc:299] Negotiation successful
0806 21:11:35.000594 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":96,"threads_started":1}
W20260806 21:11:35.001515 19105 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43632
I20260806 21:11:35.011870 19627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:35.003600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53246 (local address 127.16.178.62:39949)
0806 21:11:35.003734 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:35.003738 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:35.003833 (+ 95us) server_negotiation.cc:407] Connection header received
0806 21:11:35.003973 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:35.003978 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:35.004035 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:35.004176 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:35.004973 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:35.005551 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:35.006374 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:35.006629 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:35.006732 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:35.008225 (+ 1493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:35.008230 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:35.008232 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:35.008262 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:35.009968 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:35.010445 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:35.010449 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:35.010451 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:35.010566 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:35.010840 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:35.010843 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:35.010845 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:35.011085 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:35.011244 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:35.011581 (+ 337us) server_negotiation.cc:299] Negotiation successful
0806 21:11:35.011716 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260806 21:11:35.012529 18789 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53246
I20260806 21:11:35.014302 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 18900
I20260806 21:11:35.036688 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 19130
I20260806 21:11:35.053462 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 19345
I20260806 21:11:35.068929 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 18736
2026-08-06T21:11:35Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3184 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:11:35 dist-test-slave-lgr5 krb5kdc[19661](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:11:35 dist-test-slave-lgr5 krb5kdc[19661](info): set up 2 sockets
Aug 06 21:11:35 dist-test-slave-lgr5 krb5kdc[19661](info): commencing operation
krb5kdc: starting...
W20260806 21:11:36.227770 17096 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Aug 06 21:11:36 dist-test-slave-lgr5 krb5kdc[19661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:11:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:11:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.178.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
I20260806 21:11:36.356626 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:45481
--webserver_interface=127.16.178.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:36341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:45481
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:11:36.471055 19678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:36.471387 19678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:36.471446 19678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:36.475713 19678 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:11:36.475800 19678 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 21:11:36.475824 19678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:36.475849 19678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:36.475870 19678 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:11:36.475901 19678 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:11:36.481528 19678 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:36341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:45481
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:45481
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.178.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19678
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:36.482676 19678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:36.483716 19678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:36.492112 19690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:36.492146 19693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:36.492174 19688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:36.492203 19678 server_base.cc:1034] running on GCE node
I20260806 21:11:36.492928 19678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:36.494061 19678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:36.495236 19678 hybrid_clock.cc:648] HybridClock initialized: now 1786050696495219 us; error 31 us; skew 500 ppm
Aug 06 21:11:36 dist-test-slave-lgr5 krb5kdc[19661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050696, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:36.498281 19678 init.cc:377] Logged in from keytab as kudu/127.16.178.62@KRBTEST.COM (short username kudu)
I20260806 21:11:36.499279 19678 webserver.cc:460] Webserver started at http://127.16.178.62:43559/ using document root <none> and password file <none>
I20260806 21:11:36.499845 19678 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:36.499904 19678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:36.500161 19678 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:36.502364 19678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "a6079d59bd5d4b769d0ff72bdbd21121"
format_stamp: "Formatted at 2026-08-06 21:11:36 on dist-test-slave-lgr5"
server_key: "a5cf6b5db8c6a800a4d32412e72209db"
server_key_iv: "57e6010be3b5dbf347a9cdc9b9cf00c6"
server_key_version: "encryptionkey@0"
I20260806 21:11:36.502926 19678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "a6079d59bd5d4b769d0ff72bdbd21121"
format_stamp: "Formatted at 2026-08-06 21:11:36 on dist-test-slave-lgr5"
server_key: "a5cf6b5db8c6a800a4d32412e72209db"
server_key_iv: "57e6010be3b5dbf347a9cdc9b9cf00c6"
server_key_version: "encryptionkey@0"
I20260806 21:11:36.506676 19678 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 21:11:36.509076 19701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:36.510237 19678 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 21:11:36.510351 19678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "a6079d59bd5d4b769d0ff72bdbd21121"
format_stamp: "Formatted at 2026-08-06 21:11:36 on dist-test-slave-lgr5"
server_key: "a5cf6b5db8c6a800a4d32412e72209db"
server_key_iv: "57e6010be3b5dbf347a9cdc9b9cf00c6"
server_key_version: "encryptionkey@0"
I20260806 21:11:36.510456 19678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:36.541234 19678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:36.544607 19678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:36.544796 19678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:36.553126 19678 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.62:45481
I20260806 21:11:36.553176 19792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.62:45481 every 8 connection(s)
I20260806 21:11:36.554141 19678 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:11:36.554307 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 19678
I20260806 21:11:36.554414 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:11:36.554665 17096 external_mini_cluster.cc:1428] Setting key 8fe5417792ec822a8ef90e38cd0823f1
I20260806 21:11:36.557619 19794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 21:11:36 dist-test-slave-lgr5 krb5kdc[19661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:11:36.564224 19794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6079d59bd5d4b769d0ff72bdbd21121: Bootstrap starting.
I20260806 21:11:36.566951 19794 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a6079d59bd5d4b769d0ff72bdbd21121: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:36.568509 19794 log.cc:824] T 00000000000000000000000000000000 P a6079d59bd5d4b769d0ff72bdbd21121: Log is configured to *not* fsync() on all Append() calls
I20260806 21:11:36.569964 19797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:36.556180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54194 (local address 127.16.178.62:45481)
0806 21:11:36.556630 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:36.556649 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:36.556693 (+ 44us) server_negotiation.cc:407] Connection header received
0806 21:11:36.557520 (+ 827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:36.557543 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:36.557946 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:36.558304 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:36.558996 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:36.560475 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:36.561354 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:36.562014 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:36.562172 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:36.564781 (+ 2609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:36.564788 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:36.564803 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:36.564836 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:36.566775 (+ 1939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:36.567248 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:36.567255 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:36.567262 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:36.567405 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:36.567653 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:36.567656 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:36.567657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:36.568295 (+ 638us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:36.568493 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:36.568768 (+ 275us) server_negotiation.cc:299] Negotiation successful
0806 21:11:36.569072 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":125,"threads_started":1}
I20260806 21:11:36.570894 19794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6079d59bd5d4b769d0ff72bdbd21121: No bootstrap required, opened a new log
I20260806 21:11:36.574121 19794 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6079d59bd5d4b769d0ff72bdbd21121 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6079d59bd5d4b769d0ff72bdbd21121" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 45481 } }
I20260806 21:11:36.574370 19794 raft_consensus.cc:374] T 00000000000000000000000000000000 P a6079d59bd5d4b769d0ff72bdbd21121 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:36.574407 19794 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6079d59bd5d4b769d0ff72bdbd21121 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6079d59bd5d4b769d0ff72bdbd21121, State: Initialized, Role: FOLLOWER
I20260806 21:11:36.574849 19794 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6079d59bd5d4b769d0ff72bdbd21121 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6079d59bd5d4b769d0ff72bdbd21121" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 45481 } }
I20260806 21:11:36.574981 19794 raft_consensus.cc:388] T 00000000000000000000000000000000 P a6079d59bd5d4b769d0ff72bdbd21121 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:11:36.575068 19794 raft_consensus.cc:482] T 00000000000000000000000000000000 P a6079d59bd5d4b769d0ff72bdbd21121 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:11:36.575171 19794 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6079d59bd5d4b769d0ff72bdbd21121 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:36.576143 19794 raft_consensus.cc:504] T 00000000000000000000000000000000 P a6079d59bd5d4b769d0ff72bdbd21121 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6079d59bd5d4b769d0ff72bdbd21121" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 45481 } }
I20260806 21:11:36.576483 19794 leader_election.cc:302] T 00000000000000000000000000000000 P a6079d59bd5d4b769d0ff72bdbd21121 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a6079d59bd5d4b769d0ff72bdbd21121; no voters:
I20260806 21:11:36.576836 19794 leader_election.cc:288] T 00000000000000000000000000000000 P a6079d59bd5d4b769d0ff72bdbd21121 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:11:36.576961 19800 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a6079d59bd5d4b769d0ff72bdbd21121 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:11:36.577193 19800 raft_consensus.cc:686] T 00000000000000000000000000000000 P a6079d59bd5d4b769d0ff72bdbd21121 [term 1 LEADER]: Becoming Leader. State: Replica: a6079d59bd5d4b769d0ff72bdbd21121, State: Running, Role: LEADER
I20260806 21:11:36.577524 19800 consensus_queue.cc:237] T 00000000000000000000000000000000 P a6079d59bd5d4b769d0ff72bdbd21121 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6079d59bd5d4b769d0ff72bdbd21121" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 45481 } }
I20260806 21:11:36.578127 19794 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6079d59bd5d4b769d0ff72bdbd21121 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:11:36.579144 19802 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6079d59bd5d4b769d0ff72bdbd21121 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a6079d59bd5d4b769d0ff72bdbd21121" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6079d59bd5d4b769d0ff72bdbd21121" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 45481 } } }
I20260806 21:11:36.579277 19802 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6079d59bd5d4b769d0ff72bdbd21121 [sys.catalog]: This master's current role is: LEADER
I20260806 21:11:36.579562 19804 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6079d59bd5d4b769d0ff72bdbd21121 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a6079d59bd5d4b769d0ff72bdbd21121. Latest consensus state: current_term: 1 leader_uuid: "a6079d59bd5d4b769d0ff72bdbd21121" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6079d59bd5d4b769d0ff72bdbd21121" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 45481 } } }
I20260806 21:11:36.579653 19804 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6079d59bd5d4b769d0ff72bdbd21121 [sys.catalog]: This master's current role is: LEADER
I20260806 21:11:36.579754 19815 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:11:36.582923 19815 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:11:36.589681 19815 catalog_manager.cc:1347] Generated new cluster ID: c367c893b569492f99525fdb39648ff0
I20260806 21:11:36.589782 19815 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:11:36.597846 19815 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:11:36.598825 19815 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:11:36.603845 19815 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a6079d59bd5d4b769d0ff72bdbd21121: Generated new TSK 0
I20260806 21:11:36.604584 19815 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.178.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab.
Entry for principal kudu/127.16.178.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab.
I20260806 21:11:36.646670 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.1:0
--local_ip_for_outbound_sockets=127.16.178.1
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:45481
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab
--principal=kudu/127.16.178.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:36341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:11:36.772179 19834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:36.772488 19834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:36.772547 19834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:36.776716 19834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:36.776793 19834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:36.776832 19834 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 21:11:36.776849 19834 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 21:11:36.776901 19834 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.1
I20260806 21:11:36.782400 19834 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:36341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab
--principal=kudu/127.16.178.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.178.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.178.62:45481
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19834
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.1
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:36.783532 19834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:36.784765 19834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:36.796020 19842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:36.795997 19841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:36.796000 19846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:36.796066 19834 server_base.cc:1034] running on GCE node
I20260806 21:11:36.797420 19834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:36.798146 19834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:36.799315 19834 hybrid_clock.cc:648] HybridClock initialized: now 1786050696799305 us; error 39 us; skew 500 ppm
Aug 06 21:11:36 dist-test-slave-lgr5 krb5kdc[19661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050696, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:36.802599 19834 init.cc:377] Logged in from keytab as kudu/127.16.178.1@KRBTEST.COM (short username kudu)
I20260806 21:11:36.803612 19834 webserver.cc:460] Webserver started at http://127.16.178.1:37431/ using document root <none> and password file <none>
I20260806 21:11:36.804226 19834 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:36.804301 19834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:36.804525 19834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:36.806582 19834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-0/data/instance:
uuid: "b832bfe3f9af4b40bb1a84d3d2669c01"
format_stamp: "Formatted at 2026-08-06 21:11:36 on dist-test-slave-lgr5"
server_key: "3c5927af34f9e151ac922e43980b5572"
server_key_iv: "5e4e1047f16abe28966392a47d3972e5"
server_key_version: "encryptionkey@0"
I20260806 21:11:36.807112 19834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance:
uuid: "b832bfe3f9af4b40bb1a84d3d2669c01"
format_stamp: "Formatted at 2026-08-06 21:11:36 on dist-test-slave-lgr5"
server_key: "3c5927af34f9e151ac922e43980b5572"
server_key_iv: "5e4e1047f16abe28966392a47d3972e5"
server_key_version: "encryptionkey@0"
I20260806 21:11:36.810966 19834 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260806 21:11:36.813431 19854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:36.814742 19834 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 21:11:36.814867 19834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-0/wal
uuid: "b832bfe3f9af4b40bb1a84d3d2669c01"
format_stamp: "Formatted at 2026-08-06 21:11:36 on dist-test-slave-lgr5"
server_key: "3c5927af34f9e151ac922e43980b5572"
server_key_iv: "5e4e1047f16abe28966392a47d3972e5"
server_key_version: "encryptionkey@0"
I20260806 21:11:36.814985 19834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:36.831944 19834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:36.835178 19834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:36.835367 19834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:36.836941 19834 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:11:36.836999 19834 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:36.837054 19834 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:11:36.837081 19834 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 21:11:36 dist-test-slave-lgr5 krb5kdc[19661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050696, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:11:36.847626 19834 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.1:35443
I20260806 21:11:36.847698 20025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.1:35443 every 8 connection(s)
I20260806 21:11:36.848769 19834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
I20260806 21:11:36.851117 19797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:36.837771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:38429 (local address 127.16.178.62:45481)
0806 21:11:36.837896 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:36.837899 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:36.838936 (+ 1037us) server_negotiation.cc:407] Connection header received
0806 21:11:36.840023 (+ 1087us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:36.840036 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:36.840130 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:36.840213 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:36.841653 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:36.842469 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:36.844141 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:36.844512 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:36.844640 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:36.847546 (+ 2906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:36.847550 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:36.847553 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:36.847585 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:36.849179 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:36.849800 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:36.849803 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:36.849804 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:36.849887 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:36.850195 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:36.850197 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:36.850198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:36.850376 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:36.850442 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:36.850873 (+ 431us) server_negotiation.cc:299] Negotiation successful
0806 21:11:36.850992 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 21:11:36.851903 19864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:36.838280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:45481 (local address 127.16.178.1:38429)
0806 21:11:36.838785 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:36.838830 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:36.839743 (+ 913us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:36.840396 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:36.840406 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:36.840885 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:36.841507 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:36.841527 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:36.842586 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:36.842589 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:36.843989 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:36.844001 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:36.844685 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:36.844690 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:36.844862 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:36.845498 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:36.845515 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:36.847304 (+ 1789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:36.849294 (+ 1990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:36.849300 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:36.849315 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:36.849667 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:36.849982 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:36.849984 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:36.849986 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:36.850122 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:36.850452 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:36.850456 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:36.850685 (+ 229us) client_negotiation.cc:769] Sending connection context
0806 21:11:36.850837 (+ 152us) client_negotiation.cc:240] Negotiation successful
0806 21:11:36.851102 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":140,"threads_started":1}
I20260806 21:11:36.853286 20026 heartbeater.cc:347] Connected to a master server at 127.16.178.62:45481
I20260806 21:11:36.853453 20026 heartbeater.cc:464] Registering TS with master...
I20260806 21:11:36.854025 20026 heartbeater.cc:511] Master 127.16.178.62:45481 requested a full tablet report, sending...
I20260806 21:11:36.854861 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 19834
I20260806 21:11:36.854998 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance
I20260806 21:11:36.855332 17096 external_mini_cluster.cc:1428] Setting key 16730d851ed3cb7b86b80469b2217f58
I20260806 21:11:36.855866 19713 ts_manager.cc:194] Registered new tserver with Master: b832bfe3f9af4b40bb1a84d3d2669c01 (127.16.178.1:35443)
I20260806 21:11:36.857378 19713 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.1@KRBTEST.COM'} at 127.16.178.1:38429
WARNING: no policy specified for kudu/127.16.178.2@KRBTEST.COM; defaulting to no policy
I20260806 21:11:36.865813 20035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:36.855418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:45481 (local address 127.16.178.1:39925)
0806 21:11:36.858275 (+ 2857us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:36.858291 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:36.858403 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:36.858660 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:36.858663 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:36.858907 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:36.859078 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:36.859087 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:36.860059 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:36.860062 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:36.860662 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:36.860672 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:36.861122 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:36.861126 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:36.861217 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:36.861874 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:36.861887 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:36.862251 (+ 364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:36.864031 (+ 1780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:36.864036 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:36.864038 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:36.864302 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:36.864586 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:36.864589 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:36.864591 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:36.864678 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:36.865129 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:36.865132 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:36.865238 (+ 106us) client_negotiation.cc:769] Sending connection context
0806 21:11:36.865324 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 21:11:36.865506 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2765,"mutex_wait_us":1,"thread_start_us":129,"threads_started":1}
I20260806 21:11:36.865793 19797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:36.855463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:39925 (local address 127.16.178.62:45481)
0806 21:11:36.858088 (+ 2625us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:36.858093 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:36.858365 (+ 272us) server_negotiation.cc:407] Connection header received
0806 21:11:36.858492 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:36.858495 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:36.858546 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:36.858632 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:36.859196 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:36.859923 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:36.860777 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:36.861013 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:36.861101 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:36.862452 (+ 1351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:36.862456 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:36.862458 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:36.862486 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:36.863892 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:36.864404 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:36.864408 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:36.864410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:36.864494 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:36.864773 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:36.864777 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:36.864779 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:36.865005 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:36.865127 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:36.865424 (+ 297us) server_negotiation.cc:299] Negotiation successful
0806 21:11:36.865639 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2548}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab.
Entry for principal kudu/127.16.178.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab.
I20260806 21:11:36.887491 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.2:0
--local_ip_for_outbound_sockets=127.16.178.2
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:45481
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab
--principal=kudu/127.16.178.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:36341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:11:37.007397 20039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:37.007714 20039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:37.007764 20039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:37.012032 20039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:37.012161 20039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:37.012210 20039 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 21:11:37.012226 20039 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 21:11:37.012281 20039 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.2
I20260806 21:11:37.018015 20039 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:36341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab
--principal=kudu/127.16.178.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.178.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.178.62:45481
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.2
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:37.019279 20039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:37.020418 20039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:37.030489 20051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:37.030489 20046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:37.030539 20047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:37.030812 20039 server_base.cc:1034] running on GCE node
I20260806 21:11:37.031286 20039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:37.031948 20039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:37.033115 20039 hybrid_clock.cc:648] HybridClock initialized: now 1786050697033093 us; error 32 us; skew 500 ppm
Aug 06 21:11:37 dist-test-slave-lgr5 krb5kdc[19661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050697, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:37.038713 20039 init.cc:377] Logged in from keytab as kudu/127.16.178.2@KRBTEST.COM (short username kudu)
I20260806 21:11:37.039700 20039 webserver.cc:460] Webserver started at http://127.16.178.2:46107/ using document root <none> and password file <none>
I20260806 21:11:37.040350 20039 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:37.040426 20039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:37.040647 20039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:37.042805 20039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-1/data/instance:
uuid: "7bceff25285a4b1a818daf657bb0b467"
format_stamp: "Formatted at 2026-08-06 21:11:37 on dist-test-slave-lgr5"
server_key: "6c9dbb27f7e328d159315bd427494739"
server_key_iv: "ba2ef8c71d4dcba8da8081ba2957b5d1"
server_key_version: "encryptionkey@0"
I20260806 21:11:37.043481 20039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance:
uuid: "7bceff25285a4b1a818daf657bb0b467"
format_stamp: "Formatted at 2026-08-06 21:11:37 on dist-test-slave-lgr5"
server_key: "6c9dbb27f7e328d159315bd427494739"
server_key_iv: "ba2ef8c71d4dcba8da8081ba2957b5d1"
server_key_version: "encryptionkey@0"
I20260806 21:11:37.047467 20039 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 21:11:37.050189 20061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:37.051564 20039 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:11:37.051707 20039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-1/wal
uuid: "7bceff25285a4b1a818daf657bb0b467"
format_stamp: "Formatted at 2026-08-06 21:11:37 on dist-test-slave-lgr5"
server_key: "6c9dbb27f7e328d159315bd427494739"
server_key_iv: "ba2ef8c71d4dcba8da8081ba2957b5d1"
server_key_version: "encryptionkey@0"
I20260806 21:11:37.051842 20039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:37.071035 20039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:37.073995 20039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:37.074186 20039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:37.075675 20039 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:11:37.075726 20039 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:37.075778 20039 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:11:37.075806 20039 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 21:11:37 dist-test-slave-lgr5 krb5kdc[19661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050697, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.2@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:11:37.086536 20039 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.2:39749
I20260806 21:11:37.087352 20231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.2:39749 every 8 connection(s)
I20260806 21:11:37.087636 20039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
I20260806 21:11:37.089327 19797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:37.076570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:39335 (local address 127.16.178.62:45481)
0806 21:11:37.076689 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:37.076693 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:37.077712 (+ 1019us) server_negotiation.cc:407] Connection header received
0806 21:11:37.079030 (+ 1318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:37.079033 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:37.079093 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:37.079167 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:37.080548 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.081062 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.082279 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.082500 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.082601 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:37.085552 (+ 2951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:37.085556 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:37.085559 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:37.085589 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:37.087137 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:37.087700 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:37.087704 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:37.087706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:37.087798 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:37.088229 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:37.088232 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:37.088234 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:37.088477 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:37.088573 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:37.089072 (+ 499us) server_negotiation.cc:299] Negotiation successful
0806 21:11:37.089177 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 21:11:37.090085 20072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:37.077065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:45481 (local address 127.16.178.2:39335)
0806 21:11:37.077565 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:37.077607 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:37.078504 (+ 897us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:37.079360 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:37.079369 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:37.079837 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:37.080407 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:37.080422 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.081178 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.081181 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:37.082163 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:37.082173 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.082667 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.082672 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:37.082889 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:37.083559 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:37.083581 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:37.085378 (+ 1797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:37.087270 (+ 1892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:37.087275 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:37.087291 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:37.087596 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:37.087906 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:37.087909 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:37.087911 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:37.088113 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:37.088588 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:37.088594 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:37.088840 (+ 246us) client_negotiation.cc:769] Sending connection context
0806 21:11:37.088998 (+ 158us) client_negotiation.cc:240] Negotiation successful
0806 21:11:37.089295 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":143,"threads_started":1}
I20260806 21:11:37.091313 20233 heartbeater.cc:347] Connected to a master server at 127.16.178.62:45481
I20260806 21:11:37.091457 20233 heartbeater.cc:464] Registering TS with master...
I20260806 21:11:37.091938 20233 heartbeater.cc:511] Master 127.16.178.62:45481 requested a full tablet report, sending...
I20260806 21:11:37.092972 19725 ts_manager.cc:194] Registered new tserver with Master: 7bceff25285a4b1a818daf657bb0b467 (127.16.178.2:39749)
I20260806 21:11:37.093609 19725 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.2@KRBTEST.COM'} at 127.16.178.2:39335
I20260806 21:11:37.095369 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 20039
I20260806 21:11:37.095494 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance
I20260806 21:11:37.095748 17096 external_mini_cluster.cc:1428] Setting key 46b7910dddc902fb731b71fe0d636d13
I20260806 21:11:37.102574 19797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:37.093667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:59195 (local address 127.16.178.62:45481)
0806 21:11:37.093780 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:37.093784 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:37.094016 (+ 232us) server_negotiation.cc:407] Connection header received
0806 21:11:37.094112 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:37.094115 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:37.094154 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:37.094215 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:37.094668 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.095180 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.096022 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.096300 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.096402 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:37.099430 (+ 3028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:37.099434 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:37.099437 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:37.099474 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:37.100957 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:37.101365 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:37.101368 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:37.101370 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:37.101450 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:37.101692 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:37.101695 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:37.101697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:37.101935 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:37.102025 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:37.102339 (+ 314us) server_negotiation.cc:299] Negotiation successful
0806 21:11:37.102446 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 21:11:37.102579 20243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:37.093582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:45481 (local address 127.16.178.2:59195)
0806 21:11:37.093935 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:37.093946 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:37.094040 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:37.094242 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:37.094245 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:37.094432 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:37.094559 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:37.094566 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.095296 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.095299 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:37.095910 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:37.095919 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.096440 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.096444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:37.096548 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:37.098952 (+ 2404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:37.098964 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:37.099277 (+ 313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:37.101061 (+ 1784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:37.101064 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:37.101066 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:37.101275 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:37.101531 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:37.101534 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:37.101536 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:37.101614 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:37.102029 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:37.102033 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:37.102142 (+ 109us) client_negotiation.cc:769] Sending connection context
0806 21:11:37.102227 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 21:11:37.102381 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"mutex_wait_us":1,"thread_start_us":91,"threads_started":1}
WARNING: no policy specified for kudu/127.16.178.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab.
Entry for principal kudu/127.16.178.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab.
I20260806 21:11:37.127951 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.3:0
--local_ip_for_outbound_sockets=127.16.178.3
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:45481
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab
--principal=kudu/127.16.178.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:36341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:11:37.246897 20247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:37.247175 20247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:37.247221 20247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:37.251192 20247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:37.251255 20247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:37.251293 20247 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 21:11:37.251313 20247 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 21:11:37.251344 20247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.3
I20260806 21:11:37.256860 20247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:36341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab
--principal=kudu/127.16.178.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.178.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.178.62:45481
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.3
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:37.257948 20247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:37.258966 20247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:37.269495 20258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:37.269534 20253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:37.269495 20254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:37.269553 20247 server_base.cc:1034] running on GCE node
I20260806 21:11:37.270174 20247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:37.270890 20247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:37.272069 20247 hybrid_clock.cc:648] HybridClock initialized: now 1786050697272049 us; error 32 us; skew 500 ppm
Aug 06 21:11:37 dist-test-slave-lgr5 krb5kdc[19661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050697, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:37.275015 20247 init.cc:377] Logged in from keytab as kudu/127.16.178.3@KRBTEST.COM (short username kudu)
I20260806 21:11:37.276070 20247 webserver.cc:460] Webserver started at http://127.16.178.3:40319/ using document root <none> and password file <none>
I20260806 21:11:37.276717 20247 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:37.276782 20247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:37.276986 20247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:37.279248 20247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-2/data/instance:
uuid: "947641bfc7f24bef9d652d2bded18339"
format_stamp: "Formatted at 2026-08-06 21:11:37 on dist-test-slave-lgr5"
server_key: "edf9532b75050feee2d63c0f43e8bede"
server_key_iv: "6435cb8f26cb0b501b7e20e712e7a944"
server_key_version: "encryptionkey@0"
I20260806 21:11:37.279811 20247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance:
uuid: "947641bfc7f24bef9d652d2bded18339"
format_stamp: "Formatted at 2026-08-06 21:11:37 on dist-test-slave-lgr5"
server_key: "edf9532b75050feee2d63c0f43e8bede"
server_key_iv: "6435cb8f26cb0b501b7e20e712e7a944"
server_key_version: "encryptionkey@0"
I20260806 21:11:37.283870 20247 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260806 21:11:37.286773 20271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:37.288195 20247 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:11:37.288327 20247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-2/wal
uuid: "947641bfc7f24bef9d652d2bded18339"
format_stamp: "Formatted at 2026-08-06 21:11:37 on dist-test-slave-lgr5"
server_key: "edf9532b75050feee2d63c0f43e8bede"
server_key_iv: "6435cb8f26cb0b501b7e20e712e7a944"
server_key_version: "encryptionkey@0"
I20260806 21:11:37.288465 20247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:37.333525 20247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:37.336607 20247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:37.336793 20247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:37.338282 20247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:11:37.338342 20247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:37.338384 20247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:11:37.338397 20247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 21:11:37 dist-test-slave-lgr5 krb5kdc[19661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050697, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.3@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:11:37.349536 20247 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.3:42075
I20260806 21:11:37.350365 20442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.3:42075 every 8 connection(s)
I20260806 21:11:37.350773 20247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
I20260806 21:11:37.352486 19797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:37.339145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:39213 (local address 127.16.178.62:45481)
0806 21:11:37.339258 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:37.339262 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:37.340529 (+ 1267us) server_negotiation.cc:407] Connection header received
0806 21:11:37.341492 (+ 963us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:37.341496 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:37.341561 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:37.341620 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:37.343032 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.343575 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.344953 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.345251 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.345353 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:37.348508 (+ 3155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:37.348512 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:37.348514 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:37.348543 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:37.350140 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:37.350715 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:37.350720 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:37.350722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:37.350843 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:37.351213 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:37.351217 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:37.351219 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:37.351470 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:37.351573 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:37.352181 (+ 608us) server_negotiation.cc:299] Negotiation successful
0806 21:11:37.352318 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 21:11:37.353344 20284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:37.339695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:45481 (local address 127.16.178.3:39213)
0806 21:11:37.340367 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:37.340411 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:37.341229 (+ 818us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:37.342000 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:37.342006 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:37.342346 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:37.342876 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:37.342897 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.343817 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.343821 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:37.344824 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:37.344835 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.345423 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.345428 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:37.345634 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:37.346310 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:37.346326 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:37.348293 (+ 1967us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:37.350257 (+ 1964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:37.350262 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:37.350274 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:37.350592 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:37.350955 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:37.350957 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:37.350959 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:37.351097 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:37.351572 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:37.351580 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:37.351895 (+ 315us) client_negotiation.cc:769] Sending connection context
0806 21:11:37.352073 (+ 178us) client_negotiation.cc:240] Negotiation successful
0806 21:11:37.352455 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":182,"threads_started":1}
I20260806 21:11:37.354671 20443 heartbeater.cc:347] Connected to a master server at 127.16.178.62:45481
I20260806 21:11:37.354847 20443 heartbeater.cc:464] Registering TS with master...
I20260806 21:11:37.355420 20443 heartbeater.cc:511] Master 127.16.178.62:45481 requested a full tablet report, sending...
I20260806 21:11:37.356189 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 20247
I20260806 21:11:37.356317 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance
I20260806 21:11:37.356667 17096 external_mini_cluster.cc:1428] Setting key c7d379015f2f25c4c8fc162569c294f4
I20260806 21:11:37.356750 19725 ts_manager.cc:194] Registered new tserver with Master: 947641bfc7f24bef9d652d2bded18339 (127.16.178.3:42075)
I20260806 21:11:37.357533 19725 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.3@KRBTEST.COM'} at 127.16.178.3:39213
I20260806 21:11:37.358947 17096 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 21:11:37.364598 19797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:37.357193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:41985 (local address 127.16.178.62:45481)
0806 21:11:37.357324 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:37.357330 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:37.357548 (+ 218us) server_negotiation.cc:407] Connection header received
0806 21:11:37.357684 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:37.357687 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:37.357732 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:37.357771 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:37.358379 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.358996 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.360063 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.360379 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.360480 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:37.361570 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:37.361575 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:37.361578 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:37.361607 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:37.362876 (+ 1269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:37.363324 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:37.363328 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:37.363330 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:37.363423 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:37.363699 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:37.363702 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:37.363704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:37.363917 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:37.364024 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:37.364329 (+ 305us) server_negotiation.cc:299] Negotiation successful
0806 21:11:37.364451 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 21:11:37.364609 20450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:37.357121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:45481 (local address 127.16.178.3:41985)
0806 21:11:37.357468 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:37.357482 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:37.357578 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:37.357912 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:37.357914 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:37.358143 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:37.358266 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:37.358273 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.359138 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.359142 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:37.359931 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:37.359942 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.360482 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.360484 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:37.360558 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:37.361077 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:37.361089 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:37.361425 (+ 336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:37.362986 (+ 1561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:37.362990 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:37.362992 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:37.363193 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:37.363517 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:37.363521 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:37.363523 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:37.363610 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:37.364012 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:37.364015 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:37.364138 (+ 123us) client_negotiation.cc:769] Sending connection context
0806 21:11:37.364249 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 21:11:37.364400 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":95,"threads_started":1}
Aug 06 21:11:37 dist-test-slave-lgr5 krb5kdc[19661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 21:11:37 dist-test-slave-lgr5 krb5kdc[19661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:11:37.386041 19797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:37.378202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54206 (local address 127.16.178.62:45481)
0806 21:11:37.378339 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:37.378343 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:37.378356 (+ 13us) server_negotiation.cc:407] Connection header received
0806 21:11:37.378400 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:37.378402 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:37.378442 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:37.378518 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:37.379252 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.379817 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.380701 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.380909 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.380986 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:37.383409 (+ 2423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:37.383413 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:37.383415 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:37.383444 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:37.384682 (+ 1238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:37.385068 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:37.385071 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:37.385072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:37.385140 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:37.385347 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:37.385349 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:37.385350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:37.385519 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:37.385615 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:37.385813 (+ 198us) server_negotiation.cc:299] Negotiation successful
0806 21:11:37.385907 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 21:11:37.389070 19725 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54206:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:11:37.392472 19725 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:11:37.407956 20464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:37.401502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:35443 (local address 127.0.0.1:42664)
0806 21:11:37.401912 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:37.401972 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:37.402123 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:37.402933 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:37.402940 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:37.402968 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:11:37.403126 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:37.403135 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.404729 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.404732 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:37.406028 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:37.406038 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.407261 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.407265 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:37.407435 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:37.407505 (+ 70us) client_negotiation.cc:769] Sending connection context
0806 21:11:37.407614 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 21:11:37.407773 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"mutex_wait_us":100,"thread_start_us":90,"threads_started":1}
I20260806 21:11:37.408064 20468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:37.402528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40140 (local address 127.16.178.2:39749)
0806 21:11:37.403130 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:37.403141 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:37.403163 (+ 22us) server_negotiation.cc:407] Connection header received
0806 21:11:37.403233 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:37.403239 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:37.403442 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:37.403579 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:11:37.403815 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.404745 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.406298 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.407216 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.407309 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:37.407443 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:37.407806 (+ 363us) server_negotiation.cc:299] Negotiation successful
0806 21:11:37.407911 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":112,"threads_started":1}
I20260806 21:11:37.408398 20470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:37.402741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.3:42075 (local address 127.0.0.1:53238)
0806 21:11:37.403216 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:37.403229 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:37.403311 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:37.403975 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:37.403978 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:37.403998 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:11:37.404124 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:37.404131 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.405474 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.405490 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:37.406495 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:37.406504 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.407275 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.407278 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:37.407440 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:37.407504 (+ 64us) client_negotiation.cc:769] Sending connection context
0806 21:11:37.407611 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 21:11:37.408269 (+ 658us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"mutex_wait_us":543,"thread_start_us":101,"threads_started":1}
I20260806 21:11:37.408423 20465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:37.401654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42664 (local address 127.16.178.1:35443)
0806 21:11:37.401976 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:37.401983 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:37.402089 (+ 106us) server_negotiation.cc:407] Connection header received
0806 21:11:37.402486 (+ 397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:37.402493 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:37.402726 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:37.403296 (+ 570us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:11:37.403451 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.404608 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.406148 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.407119 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.407247 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:37.407421 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:37.408186 (+ 765us) server_negotiation.cc:299] Negotiation successful
0806 21:11:37.408292 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":103,"threads_started":1}
I20260806 21:11:37.408800 20467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:37.402383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.2:39749 (local address 127.0.0.1:40140)
0806 21:11:37.402826 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:37.402838 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:37.402935 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:37.403576 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:37.403579 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:37.403596 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:11:37.403715 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:37.403721 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.404871 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.404875 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:37.406141 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:37.406153 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.407517 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.407520 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:37.407619 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:37.407645 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 21:11:37.407713 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 21:11:37.408676 (+ 963us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"mutex_wait_us":487,"thread_start_us":127,"threads_started":1}
I20260806 21:11:37.409051 20471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:37.403165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53238 (local address 127.16.178.3:42075)
0806 21:11:37.403573 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:37.403584 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:37.403604 (+ 20us) server_negotiation.cc:407] Connection header received
0806 21:11:37.403671 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:37.403675 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:37.403845 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:37.403984 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:11:37.404235 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.405341 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.406613 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.407153 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.407772 (+ 619us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:37.407931 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:37.407983 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 21:11:37.408910 (+ 927us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":112,"threads_started":1}
I20260806 21:11:37.413118 19928 tablet_service.cc:1467] Processing CreateTablet for tablet 8b486c5910c84d68a3df80962feba997 (DEFAULT_TABLE table=test-table [id=cdebba34b80c436e970ffb37249924c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:11:37.414458 19928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b486c5910c84d68a3df80962feba997. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:37.414168 20342 tablet_service.cc:1467] Processing CreateTablet for tablet 8b486c5910c84d68a3df80962feba997 (DEFAULT_TABLE table=test-table [id=cdebba34b80c436e970ffb37249924c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:11:37.415372 20342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b486c5910c84d68a3df80962feba997. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:37.414952 20134 tablet_service.cc:1467] Processing CreateTablet for tablet 8b486c5910c84d68a3df80962feba997 (DEFAULT_TABLE table=test-table [id=cdebba34b80c436e970ffb37249924c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:11:37.416385 20134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b486c5910c84d68a3df80962feba997. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:37.423280 20472 tablet_bootstrap.cc:492] T 8b486c5910c84d68a3df80962feba997 P b832bfe3f9af4b40bb1a84d3d2669c01: Bootstrap starting.
I20260806 21:11:37.424237 20473 tablet_bootstrap.cc:492] T 8b486c5910c84d68a3df80962feba997 P 7bceff25285a4b1a818daf657bb0b467: Bootstrap starting.
I20260806 21:11:37.426115 20475 tablet_bootstrap.cc:492] T 8b486c5910c84d68a3df80962feba997 P 947641bfc7f24bef9d652d2bded18339: Bootstrap starting.
I20260806 21:11:37.427258 20472 tablet_bootstrap.cc:654] T 8b486c5910c84d68a3df80962feba997 P b832bfe3f9af4b40bb1a84d3d2669c01: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:37.427518 20473 tablet_bootstrap.cc:654] T 8b486c5910c84d68a3df80962feba997 P 7bceff25285a4b1a818daf657bb0b467: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:37.428257 20472 log.cc:824] T 8b486c5910c84d68a3df80962feba997 P b832bfe3f9af4b40bb1a84d3d2669c01: Log is configured to *not* fsync() on all Append() calls
I20260806 21:11:37.428328 20475 tablet_bootstrap.cc:654] T 8b486c5910c84d68a3df80962feba997 P 947641bfc7f24bef9d652d2bded18339: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:37.428342 20473 log.cc:824] T 8b486c5910c84d68a3df80962feba997 P 7bceff25285a4b1a818daf657bb0b467: Log is configured to *not* fsync() on all Append() calls
I20260806 21:11:37.429162 20475 log.cc:824] T 8b486c5910c84d68a3df80962feba997 P 947641bfc7f24bef9d652d2bded18339: Log is configured to *not* fsync() on all Append() calls
I20260806 21:11:37.430652 20472 tablet_bootstrap.cc:492] T 8b486c5910c84d68a3df80962feba997 P b832bfe3f9af4b40bb1a84d3d2669c01: No bootstrap required, opened a new log
I20260806 21:11:37.430773 20473 tablet_bootstrap.cc:492] T 8b486c5910c84d68a3df80962feba997 P 7bceff25285a4b1a818daf657bb0b467: No bootstrap required, opened a new log
I20260806 21:11:37.430899 20472 ts_tablet_manager.cc:1397] T 8b486c5910c84d68a3df80962feba997 P b832bfe3f9af4b40bb1a84d3d2669c01: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.002s
I20260806 21:11:37.430967 20473 ts_tablet_manager.cc:1397] T 8b486c5910c84d68a3df80962feba997 P 7bceff25285a4b1a818daf657bb0b467: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 21:11:37.431228 20475 tablet_bootstrap.cc:492] T 8b486c5910c84d68a3df80962feba997 P 947641bfc7f24bef9d652d2bded18339: No bootstrap required, opened a new log
I20260806 21:11:37.431404 20475 ts_tablet_manager.cc:1397] T 8b486c5910c84d68a3df80962feba997 P 947641bfc7f24bef9d652d2bded18339: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 21:11:37.434175 20473 raft_consensus.cc:348] T 8b486c5910c84d68a3df80962feba997 P 7bceff25285a4b1a818daf657bb0b467 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } }
I20260806 21:11:37.434427 20473 raft_consensus.cc:374] T 8b486c5910c84d68a3df80962feba997 P 7bceff25285a4b1a818daf657bb0b467 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:37.434492 20473 raft_consensus.cc:729] T 8b486c5910c84d68a3df80962feba997 P 7bceff25285a4b1a818daf657bb0b467 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bceff25285a4b1a818daf657bb0b467, State: Initialized, Role: FOLLOWER
I20260806 21:11:37.434935 20473 consensus_queue.cc:260] T 8b486c5910c84d68a3df80962feba997 P 7bceff25285a4b1a818daf657bb0b467 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } }
I20260806 21:11:37.435281 20475 raft_consensus.cc:348] T 8b486c5910c84d68a3df80962feba997 P 947641bfc7f24bef9d652d2bded18339 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } }
I20260806 21:11:37.435546 20475 raft_consensus.cc:374] T 8b486c5910c84d68a3df80962feba997 P 947641bfc7f24bef9d652d2bded18339 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:37.435606 20475 raft_consensus.cc:729] T 8b486c5910c84d68a3df80962feba997 P 947641bfc7f24bef9d652d2bded18339 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 947641bfc7f24bef9d652d2bded18339, State: Initialized, Role: FOLLOWER
I20260806 21:11:37.435668 20233 heartbeater.cc:503] Master 127.16.178.62:45481 was elected leader, sending a full tablet report...
I20260806 21:11:37.436022 20473 ts_tablet_manager.cc:1428] T 8b486c5910c84d68a3df80962feba997 P 7bceff25285a4b1a818daf657bb0b467: Time spent starting tablet: real 0.005s user 0.001s sys 0.003s
I20260806 21:11:37.436023 20475 consensus_queue.cc:260] T 8b486c5910c84d68a3df80962feba997 P 947641bfc7f24bef9d652d2bded18339 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } }
I20260806 21:11:37.436301 20472 raft_consensus.cc:348] T 8b486c5910c84d68a3df80962feba997 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } }
I20260806 21:11:37.436560 20472 raft_consensus.cc:374] T 8b486c5910c84d68a3df80962feba997 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:37.436733 20443 heartbeater.cc:503] Master 127.16.178.62:45481 was elected leader, sending a full tablet report...
I20260806 21:11:37.437034 20475 ts_tablet_manager.cc:1428] T 8b486c5910c84d68a3df80962feba997 P 947641bfc7f24bef9d652d2bded18339: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 21:11:37.437346 20472 raft_consensus.cc:729] T 8b486c5910c84d68a3df80962feba997 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b832bfe3f9af4b40bb1a84d3d2669c01, State: Initialized, Role: FOLLOWER
I20260806 21:11:37.437775 20472 consensus_queue.cc:260] T 8b486c5910c84d68a3df80962feba997 P b832bfe3f9af4b40bb1a84d3d2669c01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } }
I20260806 21:11:37.438702 20472 ts_tablet_manager.cc:1428] T 8b486c5910c84d68a3df80962feba997 P b832bfe3f9af4b40bb1a84d3d2669c01: Time spent starting tablet: real 0.008s user 0.006s sys 0.002s
I20260806 21:11:37.438997 20026 heartbeater.cc:503] Master 127.16.178.62:45481 was elected leader, sending a full tablet report...
I20260806 21:11:37.461112 20482 raft_consensus.cc:482] T 8b486c5910c84d68a3df80962feba997 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:11:37.461324 20482 raft_consensus.cc:504] T 8b486c5910c84d68a3df80962feba997 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } }
I20260806 21:11:37.462570 20482 leader_election.cc:288] T 8b486c5910c84d68a3df80962feba997 P b832bfe3f9af4b40bb1a84d3d2669c01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bceff25285a4b1a818daf657bb0b467 (127.16.178.2:39749), 947641bfc7f24bef9d652d2bded18339 (127.16.178.3:42075)
I20260806 21:11:37.466712 20471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:37.463394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:40083 (local address 127.16.178.3:42075)
0806 21:11:37.463561 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:37.463565 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:37.463666 (+ 101us) server_negotiation.cc:407] Connection header received
0806 21:11:37.463807 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:37.463809 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:37.463853 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:37.463911 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:11:37.464194 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.464727 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.465774 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.466184 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.466298 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:37.466355 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:37.466517 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 21:11:37.466585 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260806 21:11:37.466768 20485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:37.463261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.3:42075 (local address 127.16.178.1:40083)
0806 21:11:37.463587 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:37.463600 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:37.463704 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:37.463931 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:37.463933 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:37.463948 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:11:37.464074 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:37.464105 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.464838 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.464842 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:37.465648 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:37.465659 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.466298 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.466302 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:37.466399 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:37.466423 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 21:11:37.466494 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 21:11:37.466567 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"mutex_wait_us":1,"spinlock_wait_cycles":768,"thread_start_us":98,"threads_started":1}
I20260806 21:11:37.466773 20468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:37.462936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:44027 (local address 127.16.178.2:39749)
0806 21:11:37.463089 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:37.463095 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:37.463226 (+ 131us) server_negotiation.cc:407] Connection header received
0806 21:11:37.463451 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:37.463455 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:37.463515 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:37.463594 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:11:37.463889 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.464453 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.465839 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.466240 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.466356 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:37.466410 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:37.466577 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 21:11:37.466649 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 21:11:37.466800 20484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:37.462800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.2:39749 (local address 127.16.178.1:44027)
0806 21:11:37.463137 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:37.463151 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:37.463273 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:37.463621 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:37.463624 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:37.463643 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:11:37.463770 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:37.463776 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.464566 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.464569 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:37.465713 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:37.465725 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.466354 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.466358 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:37.466465 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:37.466487 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 21:11:37.466553 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 21:11:37.466625 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"mutex_wait_us":37,"thread_start_us":144,"threads_started":1}
I20260806 21:11:37.467356 20374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b486c5910c84d68a3df80962feba997" candidate_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "947641bfc7f24bef9d652d2bded18339" is_pre_election: true
I20260806 21:11:37.467588 20168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b486c5910c84d68a3df80962feba997" candidate_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bceff25285a4b1a818daf657bb0b467" is_pre_election: true
I20260806 21:11:37.467677 20374 raft_consensus.cc:2454] T 8b486c5910c84d68a3df80962feba997 P 947641bfc7f24bef9d652d2bded18339 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b832bfe3f9af4b40bb1a84d3d2669c01 in term 0.
I20260806 21:11:37.467842 20168 raft_consensus.cc:2454] T 8b486c5910c84d68a3df80962feba997 P 7bceff25285a4b1a818daf657bb0b467 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b832bfe3f9af4b40bb1a84d3d2669c01 in term 0.
I20260806 21:11:37.468186 19857 leader_election.cc:302] T 8b486c5910c84d68a3df80962feba997 P b832bfe3f9af4b40bb1a84d3d2669c01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 947641bfc7f24bef9d652d2bded18339, b832bfe3f9af4b40bb1a84d3d2669c01; no voters:
I20260806 21:11:37.468442 20482 raft_consensus.cc:2781] T 8b486c5910c84d68a3df80962feba997 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:11:37.468528 20482 raft_consensus.cc:482] T 8b486c5910c84d68a3df80962feba997 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:11:37.468580 20482 raft_consensus.cc:3037] T 8b486c5910c84d68a3df80962feba997 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:37.469846 20482 raft_consensus.cc:504] T 8b486c5910c84d68a3df80962feba997 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } }
I20260806 21:11:37.470213 20482 leader_election.cc:288] T 8b486c5910c84d68a3df80962feba997 P b832bfe3f9af4b40bb1a84d3d2669c01 [CANDIDATE]: Term 1 election: Requested vote from peers 7bceff25285a4b1a818daf657bb0b467 (127.16.178.2:39749), 947641bfc7f24bef9d652d2bded18339 (127.16.178.3:42075)
I20260806 21:11:37.470618 20168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b486c5910c84d68a3df80962feba997" candidate_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bceff25285a4b1a818daf657bb0b467"
I20260806 21:11:37.470731 20168 raft_consensus.cc:3037] T 8b486c5910c84d68a3df80962feba997 P 7bceff25285a4b1a818daf657bb0b467 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:37.471741 20168 raft_consensus.cc:2454] T 8b486c5910c84d68a3df80962feba997 P 7bceff25285a4b1a818daf657bb0b467 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b832bfe3f9af4b40bb1a84d3d2669c01 in term 1.
I20260806 21:11:37.472012 20374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b486c5910c84d68a3df80962feba997" candidate_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "947641bfc7f24bef9d652d2bded18339"
I20260806 21:11:37.472049 19857 leader_election.cc:302] T 8b486c5910c84d68a3df80962feba997 P b832bfe3f9af4b40bb1a84d3d2669c01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bceff25285a4b1a818daf657bb0b467, b832bfe3f9af4b40bb1a84d3d2669c01; no voters:
I20260806 21:11:37.472174 20374 raft_consensus.cc:3037] T 8b486c5910c84d68a3df80962feba997 P 947641bfc7f24bef9d652d2bded18339 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:37.472266 20482 raft_consensus.cc:2781] T 8b486c5910c84d68a3df80962feba997 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:11:37.472518 20482 raft_consensus.cc:686] T 8b486c5910c84d68a3df80962feba997 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 1 LEADER]: Becoming Leader. State: Replica: b832bfe3f9af4b40bb1a84d3d2669c01, State: Running, Role: LEADER
I20260806 21:11:37.472883 20482 consensus_queue.cc:237] T 8b486c5910c84d68a3df80962feba997 P b832bfe3f9af4b40bb1a84d3d2669c01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } }
I20260806 21:11:37.473320 20374 raft_consensus.cc:2454] T 8b486c5910c84d68a3df80962feba997 P 947641bfc7f24bef9d652d2bded18339 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b832bfe3f9af4b40bb1a84d3d2669c01 in term 1.
I20260806 21:11:37.476313 19713 catalog_manager.cc:5495] T 8b486c5910c84d68a3df80962feba997 P b832bfe3f9af4b40bb1a84d3d2669c01 reported cstate change: term changed from 0 to 1, leader changed from <none> to b832bfe3f9af4b40bb1a84d3d2669c01 (127.16.178.1). New cstate: current_term: 1 leader_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } health_report { overall_health: UNKNOWN } } }
Aug 06 21:11:37 dist-test-slave-lgr5 krb5kdc[19661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050696, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:11:37.526094 19797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:37.518083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54218 (local address 127.16.178.62:45481)
0806 21:11:37.518220 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:37.518225 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:37.518427 (+ 202us) server_negotiation.cc:407] Connection header received
0806 21:11:37.518557 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:37.518559 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:37.518597 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:37.518659 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:37.519366 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.519884 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.520722 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.520922 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.521006 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:37.523174 (+ 2168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:37.523178 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:37.523180 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:37.523204 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:37.524549 (+ 1345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:37.524982 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:37.524986 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:37.524988 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:37.525063 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:37.525298 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:37.525300 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:37.525301 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:37.525464 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:37.525554 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:37.525810 (+ 256us) server_negotiation.cc:299] Negotiation successful
0806 21:11:37.525953 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":48}
I20260806 21:11:37.526095 20495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:37.518014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:45481 (local address 127.0.0.1:54218)
0806 21:11:37.518316 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:37.518328 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:37.518463 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:37.518690 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:37.518693 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:37.519115 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:37.519257 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:37.519267 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.519996 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.519999 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:37.520610 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:37.520620 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.521024 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.521028 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:37.521100 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:37.521598 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:37.521610 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:37.523009 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:37.524652 (+ 1643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:37.524662 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:37.524668 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:37.524894 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:37.525145 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:37.525147 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:37.525148 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:37.525233 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:37.525561 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:37.525566 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:37.525676 (+ 110us) client_negotiation.cc:769] Sending connection context
0806 21:11:37.525767 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 21:11:37.525896 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":108,"threads_started":1}
I20260806 21:11:37.530332 19713 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:54218:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 21:11:37.531143 19713 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:11:37.536348 19928 tablet_service.cc:1467] Processing CreateTablet for tablet a051d1152c1e477d9555fe8fee23a186 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1bac648dc4914b1ebb90756de9cf3cb2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 21:11:37.536656 20134 tablet_service.cc:1467] Processing CreateTablet for tablet a051d1152c1e477d9555fe8fee23a186 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1bac648dc4914b1ebb90756de9cf3cb2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 21:11:37.537168 19928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a051d1152c1e477d9555fe8fee23a186. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:37.537253 20134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a051d1152c1e477d9555fe8fee23a186. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:37.538421 20342 tablet_service.cc:1467] Processing CreateTablet for tablet a051d1152c1e477d9555fe8fee23a186 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1bac648dc4914b1ebb90756de9cf3cb2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 21:11:37.539165 20342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a051d1152c1e477d9555fe8fee23a186. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:37.539856 20472 tablet_bootstrap.cc:492] T a051d1152c1e477d9555fe8fee23a186 P b832bfe3f9af4b40bb1a84d3d2669c01: Bootstrap starting.
I20260806 21:11:37.540145 20473 tablet_bootstrap.cc:492] T a051d1152c1e477d9555fe8fee23a186 P 7bceff25285a4b1a818daf657bb0b467: Bootstrap starting.
I20260806 21:11:37.540948 20472 tablet_bootstrap.cc:654] T a051d1152c1e477d9555fe8fee23a186 P b832bfe3f9af4b40bb1a84d3d2669c01: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:37.541098 20473 tablet_bootstrap.cc:654] T a051d1152c1e477d9555fe8fee23a186 P 7bceff25285a4b1a818daf657bb0b467: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:37.542428 20475 tablet_bootstrap.cc:492] T a051d1152c1e477d9555fe8fee23a186 P 947641bfc7f24bef9d652d2bded18339: Bootstrap starting.
I20260806 21:11:37.542510 20473 tablet_bootstrap.cc:492] T a051d1152c1e477d9555fe8fee23a186 P 7bceff25285a4b1a818daf657bb0b467: No bootstrap required, opened a new log
I20260806 21:11:37.542600 20473 ts_tablet_manager.cc:1397] T a051d1152c1e477d9555fe8fee23a186 P 7bceff25285a4b1a818daf657bb0b467: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 21:11:37.542654 20472 tablet_bootstrap.cc:492] T a051d1152c1e477d9555fe8fee23a186 P b832bfe3f9af4b40bb1a84d3d2669c01: No bootstrap required, opened a new log
I20260806 21:11:37.542730 20472 ts_tablet_manager.cc:1397] T a051d1152c1e477d9555fe8fee23a186 P b832bfe3f9af4b40bb1a84d3d2669c01: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 21:11:37.543094 20473 raft_consensus.cc:348] T a051d1152c1e477d9555fe8fee23a186 P 7bceff25285a4b1a818daf657bb0b467 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } }
I20260806 21:11:37.543147 20472 raft_consensus.cc:348] T a051d1152c1e477d9555fe8fee23a186 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } }
I20260806 21:11:37.543213 20473 raft_consensus.cc:374] T a051d1152c1e477d9555fe8fee23a186 P 7bceff25285a4b1a818daf657bb0b467 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:37.543236 20472 raft_consensus.cc:374] T a051d1152c1e477d9555fe8fee23a186 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:37.543244 20473 raft_consensus.cc:729] T a051d1152c1e477d9555fe8fee23a186 P 7bceff25285a4b1a818daf657bb0b467 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bceff25285a4b1a818daf657bb0b467, State: Initialized, Role: FOLLOWER
I20260806 21:11:37.543262 20475 tablet_bootstrap.cc:654] T a051d1152c1e477d9555fe8fee23a186 P 947641bfc7f24bef9d652d2bded18339: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:37.543267 20472 raft_consensus.cc:729] T a051d1152c1e477d9555fe8fee23a186 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b832bfe3f9af4b40bb1a84d3d2669c01, State: Initialized, Role: FOLLOWER
I20260806 21:11:37.543380 20472 consensus_queue.cc:260] T a051d1152c1e477d9555fe8fee23a186 P b832bfe3f9af4b40bb1a84d3d2669c01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } }
I20260806 21:11:37.543380 20473 consensus_queue.cc:260] T a051d1152c1e477d9555fe8fee23a186 P 7bceff25285a4b1a818daf657bb0b467 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } }
I20260806 21:11:37.543651 20472 ts_tablet_manager.cc:1428] T a051d1152c1e477d9555fe8fee23a186 P b832bfe3f9af4b40bb1a84d3d2669c01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 21:11:37.543651 20473 ts_tablet_manager.cc:1428] T a051d1152c1e477d9555fe8fee23a186 P 7bceff25285a4b1a818daf657bb0b467: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 21:11:37.544198 20482 tablet_replica.cc:439] T a051d1152c1e477d9555fe8fee23a186 P b832bfe3f9af4b40bb1a84d3d2669c01: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } } }
I20260806 21:11:37.544405 20482 tablet_replica.cc:442] T a051d1152c1e477d9555fe8fee23a186 P b832bfe3f9af4b40bb1a84d3d2669c01: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 21:11:37.544351 20480 tablet_replica.cc:439] T a051d1152c1e477d9555fe8fee23a186 P 7bceff25285a4b1a818daf657bb0b467: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } } }
I20260806 21:11:37.544499 20480 tablet_replica.cc:442] T a051d1152c1e477d9555fe8fee23a186 P 7bceff25285a4b1a818daf657bb0b467: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 21:11:37.545143 20475 tablet_bootstrap.cc:492] T a051d1152c1e477d9555fe8fee23a186 P 947641bfc7f24bef9d652d2bded18339: No bootstrap required, opened a new log
I20260806 21:11:37.545257 20475 ts_tablet_manager.cc:1397] T a051d1152c1e477d9555fe8fee23a186 P 947641bfc7f24bef9d652d2bded18339: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 21:11:37.545802 20475 raft_consensus.cc:348] T a051d1152c1e477d9555fe8fee23a186 P 947641bfc7f24bef9d652d2bded18339 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } }
I20260806 21:11:37.545928 20475 raft_consensus.cc:374] T a051d1152c1e477d9555fe8fee23a186 P 947641bfc7f24bef9d652d2bded18339 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:37.545966 20475 raft_consensus.cc:729] T a051d1152c1e477d9555fe8fee23a186 P 947641bfc7f24bef9d652d2bded18339 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 947641bfc7f24bef9d652d2bded18339, State: Initialized, Role: FOLLOWER
I20260806 21:11:37.546098 20475 consensus_queue.cc:260] T a051d1152c1e477d9555fe8fee23a186 P 947641bfc7f24bef9d652d2bded18339 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } }
I20260806 21:11:37.546387 20475 ts_tablet_manager.cc:1428] T a051d1152c1e477d9555fe8fee23a186 P 947641bfc7f24bef9d652d2bded18339: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 21:11:37.546650 20481 tablet_replica.cc:439] T a051d1152c1e477d9555fe8fee23a186 P 947641bfc7f24bef9d652d2bded18339: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } } }
I20260806 21:11:37.546813 20481 tablet_replica.cc:442] T a051d1152c1e477d9555fe8fee23a186 P 947641bfc7f24bef9d652d2bded18339: This TxnStatusTablet replica's current role is: FOLLOWER
W20260806 21:11:37.588976 20234 tablet.cc:2369] T 8b486c5910c84d68a3df80962feba997 P 7bceff25285a4b1a818daf657bb0b467: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 21:11:37.598066 20480 raft_consensus.cc:482] T a051d1152c1e477d9555fe8fee23a186 P 7bceff25285a4b1a818daf657bb0b467 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:11:37.598232 20480 raft_consensus.cc:504] T a051d1152c1e477d9555fe8fee23a186 P 7bceff25285a4b1a818daf657bb0b467 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } }
I20260806 21:11:37.599377 20480 leader_election.cc:288] T a051d1152c1e477d9555fe8fee23a186 P 7bceff25285a4b1a818daf657bb0b467 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b832bfe3f9af4b40bb1a84d3d2669c01 (127.16.178.1:35443), 947641bfc7f24bef9d652d2bded18339 (127.16.178.3:42075)
W20260806 21:11:37.600313 20027 tablet.cc:2369] T 8b486c5910c84d68a3df80962feba997 P b832bfe3f9af4b40bb1a84d3d2669c01: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 21:11:37.601737 20444 tablet.cc:2369] T 8b486c5910c84d68a3df80962feba997 P 947641bfc7f24bef9d652d2bded18339: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 21:11:37.603833 20502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:37.599655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:35443 (local address 127.16.178.2:47317)
0806 21:11:37.599936 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:37.599952 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:37.600117 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:37.600559 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:37.600561 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:37.600578 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:11:37.600678 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:37.600683 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.601601 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.601605 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:37.602632 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:37.602644 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.603389 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.603393 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:37.603499 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:37.603528 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 21:11:37.603613 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 21:11:37.603689 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":93,"threads_started":1}
I20260806 21:11:37.603919 20465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:37.599758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:47317 (local address 127.16.178.1:35443)
0806 21:11:37.599881 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:37.599884 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:37.600026 (+ 142us) server_negotiation.cc:407] Connection header received
0806 21:11:37.600284 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:37.600288 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:37.600358 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:37.600404 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:11:37.600788 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.601467 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.602771 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.603270 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.603378 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:37.603424 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:37.603679 (+ 255us) server_negotiation.cc:299] Negotiation successful
0806 21:11:37.603773 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 21:11:37.604409 20503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:37.600143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.3:42075 (local address 127.16.178.2:58293)
0806 21:11:37.600657 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:37.600673 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:37.600846 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:37.601140 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:37.601143 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:37.601163 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:11:37.601275 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:37.601281 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.602048 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.602052 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:37.602908 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:37.602920 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.603682 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.603687 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:37.603822 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:37.603850 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 21:11:37.604182 (+ 332us) client_negotiation.cc:240] Negotiation successful
0806 21:11:37.604265 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":83,"threads_started":1}
I20260806 21:11:37.604446 20471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:37.600302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:58293 (local address 127.16.178.3:42075)
0806 21:11:37.600434 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:37.600439 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:37.600771 (+ 332us) server_negotiation.cc:407] Connection header received
0806 21:11:37.600976 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:37.600980 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:37.601033 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:37.601111 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:11:37.601392 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.601945 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.603059 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.603531 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.603650 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:37.603695 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:37.604206 (+ 511us) server_negotiation.cc:299] Negotiation successful
0806 21:11:37.604329 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 21:11:37.604558 19955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a051d1152c1e477d9555fe8fee23a186" candidate_uuid: "7bceff25285a4b1a818daf657bb0b467" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" is_pre_election: true
I20260806 21:11:37.604825 19955 raft_consensus.cc:2454] T a051d1152c1e477d9555fe8fee23a186 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bceff25285a4b1a818daf657bb0b467 in term 0.
I20260806 21:11:37.604893 20374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a051d1152c1e477d9555fe8fee23a186" candidate_uuid: "7bceff25285a4b1a818daf657bb0b467" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "947641bfc7f24bef9d652d2bded18339" is_pre_election: true
I20260806 21:11:37.605074 20374 raft_consensus.cc:2454] T a051d1152c1e477d9555fe8fee23a186 P 947641bfc7f24bef9d652d2bded18339 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bceff25285a4b1a818daf657bb0b467 in term 0.
I20260806 21:11:37.605290 20066 leader_election.cc:302] T a051d1152c1e477d9555fe8fee23a186 P 7bceff25285a4b1a818daf657bb0b467 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bceff25285a4b1a818daf657bb0b467, b832bfe3f9af4b40bb1a84d3d2669c01; no voters:
I20260806 21:11:37.605571 20480 raft_consensus.cc:2781] T a051d1152c1e477d9555fe8fee23a186 P 7bceff25285a4b1a818daf657bb0b467 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:11:37.605638 20480 raft_consensus.cc:482] T a051d1152c1e477d9555fe8fee23a186 P 7bceff25285a4b1a818daf657bb0b467 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:11:37.605674 20480 raft_consensus.cc:3037] T a051d1152c1e477d9555fe8fee23a186 P 7bceff25285a4b1a818daf657bb0b467 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:37.606712 20480 raft_consensus.cc:504] T a051d1152c1e477d9555fe8fee23a186 P 7bceff25285a4b1a818daf657bb0b467 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } }
I20260806 21:11:37.607038 20480 leader_election.cc:288] T a051d1152c1e477d9555fe8fee23a186 P 7bceff25285a4b1a818daf657bb0b467 [CANDIDATE]: Term 1 election: Requested vote from peers b832bfe3f9af4b40bb1a84d3d2669c01 (127.16.178.1:35443), 947641bfc7f24bef9d652d2bded18339 (127.16.178.3:42075)
I20260806 21:11:37.607465 20374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a051d1152c1e477d9555fe8fee23a186" candidate_uuid: "7bceff25285a4b1a818daf657bb0b467" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "947641bfc7f24bef9d652d2bded18339"
I20260806 21:11:37.607452 19955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a051d1152c1e477d9555fe8fee23a186" candidate_uuid: "7bceff25285a4b1a818daf657bb0b467" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01"
I20260806 21:11:37.607597 20374 raft_consensus.cc:3037] T a051d1152c1e477d9555fe8fee23a186 P 947641bfc7f24bef9d652d2bded18339 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:37.607645 19955 raft_consensus.cc:3037] T a051d1152c1e477d9555fe8fee23a186 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:37.608621 20374 raft_consensus.cc:2454] T a051d1152c1e477d9555fe8fee23a186 P 947641bfc7f24bef9d652d2bded18339 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bceff25285a4b1a818daf657bb0b467 in term 1.
I20260806 21:11:37.608788 19955 raft_consensus.cc:2454] T a051d1152c1e477d9555fe8fee23a186 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bceff25285a4b1a818daf657bb0b467 in term 1.
I20260806 21:11:37.608896 20066 leader_election.cc:302] T a051d1152c1e477d9555fe8fee23a186 P 7bceff25285a4b1a818daf657bb0b467 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bceff25285a4b1a818daf657bb0b467, 947641bfc7f24bef9d652d2bded18339; no voters:
I20260806 21:11:37.609079 20480 raft_consensus.cc:2781] T a051d1152c1e477d9555fe8fee23a186 P 7bceff25285a4b1a818daf657bb0b467 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:11:37.609261 20480 raft_consensus.cc:686] T a051d1152c1e477d9555fe8fee23a186 P 7bceff25285a4b1a818daf657bb0b467 [term 1 LEADER]: Becoming Leader. State: Replica: 7bceff25285a4b1a818daf657bb0b467, State: Running, Role: LEADER
I20260806 21:11:37.609608 20480 consensus_queue.cc:237] T a051d1152c1e477d9555fe8fee23a186 P 7bceff25285a4b1a818daf657bb0b467 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } }
I20260806 21:11:37.611838 20504 tablet_replica.cc:439] T a051d1152c1e477d9555fe8fee23a186 P 7bceff25285a4b1a818daf657bb0b467: TxnStatusTablet state changed. Reason: New leader 7bceff25285a4b1a818daf657bb0b467. Latest consensus state: current_term: 1 leader_uuid: "7bceff25285a4b1a818daf657bb0b467" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } } }
I20260806 21:11:37.611959 20504 tablet_replica.cc:442] T a051d1152c1e477d9555fe8fee23a186 P 7bceff25285a4b1a818daf657bb0b467: This TxnStatusTablet replica's current role is: LEADER
I20260806 21:11:37.612279 20506 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 21:11:37.612511 20506 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 21:11:37.613560 19713 catalog_manager.cc:5495] T a051d1152c1e477d9555fe8fee23a186 P 7bceff25285a4b1a818daf657bb0b467 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7bceff25285a4b1a818daf657bb0b467 (127.16.178.2). New cstate: current_term: 1 leader_uuid: "7bceff25285a4b1a818daf657bb0b467" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } health_report { overall_health: UNKNOWN } } }
I20260806 21:11:37.634732 20495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:37.630935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.2:39749 (local address 127.0.0.1:40156)
0806 21:11:37.631205 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:37.631220 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:37.631342 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:37.631667 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:37.631671 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:37.631678 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 21:11:37.631788 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:37.631794 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.632554 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.632558 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:37.633160 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:37.633169 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.633603 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.633607 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:37.633702 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:37.633752 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 21:11:37.634435 (+ 683us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 21:11:37.634453 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 21:11:37.634519 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 21:11:37.634605 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157}
I20260806 21:11:37.634733 20468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:37.631048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40156 (local address 127.16.178.2:39749)
0806 21:11:37.631189 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:37.631194 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:37.631280 (+ 86us) server_negotiation.cc:407] Connection header received
0806 21:11:37.631489 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:37.631492 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:37.631550 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:37.631634 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:11:37.631893 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.632436 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.633282 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.633499 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.633585 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:37.633894 (+ 309us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:11:37.634334 (+ 440us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:11:37.634438 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:37.634530 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 21:11:37.634585 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 21:11:37.640115 19955 raft_consensus.cc:1261] T a051d1152c1e477d9555fe8fee23a186 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 1 FOLLOWER]: Refusing update from remote peer 7bceff25285a4b1a818daf657bb0b467: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:11:37.640194 20374 raft_consensus.cc:1261] T a051d1152c1e477d9555fe8fee23a186 P 947641bfc7f24bef9d652d2bded18339 [term 1 FOLLOWER]: Refusing update from remote peer 7bceff25285a4b1a818daf657bb0b467: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:11:37.640781 20504 consensus_queue.cc:1035] T a051d1152c1e477d9555fe8fee23a186 P 7bceff25285a4b1a818daf657bb0b467 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:11:37.640974 20480 consensus_queue.cc:1035] T a051d1152c1e477d9555fe8fee23a186 P 7bceff25285a4b1a818daf657bb0b467 [LEADER]: Connected to new peer: Peer: permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:11:37.644441 20482 tablet_replica.cc:439] T a051d1152c1e477d9555fe8fee23a186 P b832bfe3f9af4b40bb1a84d3d2669c01: TxnStatusTablet state changed. Reason: New leader 7bceff25285a4b1a818daf657bb0b467. Latest consensus state: current_term: 1 leader_uuid: "7bceff25285a4b1a818daf657bb0b467" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } } }
I20260806 21:11:37.644565 20482 tablet_replica.cc:442] T a051d1152c1e477d9555fe8fee23a186 P b832bfe3f9af4b40bb1a84d3d2669c01: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 21:11:37.645047 20481 tablet_replica.cc:439] T a051d1152c1e477d9555fe8fee23a186 P 947641bfc7f24bef9d652d2bded18339: TxnStatusTablet state changed. Reason: New leader 7bceff25285a4b1a818daf657bb0b467. Latest consensus state: current_term: 1 leader_uuid: "7bceff25285a4b1a818daf657bb0b467" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } } }
I20260806 21:11:37.645159 20481 tablet_replica.cc:442] T a051d1152c1e477d9555fe8fee23a186 P 947641bfc7f24bef9d652d2bded18339: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 21:11:37.648799 20480 tablet_replica.cc:439] T a051d1152c1e477d9555fe8fee23a186 P 7bceff25285a4b1a818daf657bb0b467: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7bceff25285a4b1a818daf657bb0b467" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } } }
I20260806 21:11:37.648929 20480 tablet_replica.cc:442] T a051d1152c1e477d9555fe8fee23a186 P 7bceff25285a4b1a818daf657bb0b467: This TxnStatusTablet replica's current role is: LEADER
I20260806 21:11:37.649323 20512 mvcc.cc:205] Tried to move back new op lower bound from 7315663657526603776 to 7315663657411616768. Current Snapshot: MvccSnapshot[applied={T|T < 7315663657526603776}]
I20260806 21:11:37.649833 20481 tablet_replica.cc:439] T a051d1152c1e477d9555fe8fee23a186 P 947641bfc7f24bef9d652d2bded18339: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7bceff25285a4b1a818daf657bb0b467" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } } }
I20260806 21:11:37.649936 20481 tablet_replica.cc:442] T a051d1152c1e477d9555fe8fee23a186 P 947641bfc7f24bef9d652d2bded18339: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 21:11:37.650645 20513 mvcc.cc:205] Tried to move back new op lower bound from 7315663657526603776 to 7315663657411616768. Current Snapshot: MvccSnapshot[applied={T|T < 7315663657526603776}]
I20260806 21:11:37.651154 20482 tablet_replica.cc:439] T a051d1152c1e477d9555fe8fee23a186 P b832bfe3f9af4b40bb1a84d3d2669c01: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7bceff25285a4b1a818daf657bb0b467" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } } }
I20260806 21:11:37.651259 20482 tablet_replica.cc:442] T a051d1152c1e477d9555fe8fee23a186 P b832bfe3f9af4b40bb1a84d3d2669c01: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 21:11:37.648809 20504 tablet_replica.cc:439] T a051d1152c1e477d9555fe8fee23a186 P 7bceff25285a4b1a818daf657bb0b467: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7bceff25285a4b1a818daf657bb0b467" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } } }
I20260806 21:11:37.652979 20504 tablet_replica.cc:442] T a051d1152c1e477d9555fe8fee23a186 P 7bceff25285a4b1a818daf657bb0b467: This TxnStatusTablet replica's current role is: LEADER
I20260806 21:11:37.661617 20465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:37.657670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42666 (local address 127.16.178.1:35443)
0806 21:11:37.657796 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:37.657809 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:37.657825 (+ 16us) server_negotiation.cc:407] Connection header received
0806 21:11:37.657934 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:37.657937 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:37.658002 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:37.658111 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:11:37.658313 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.659106 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.659939 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.660317 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.660417 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:37.660662 (+ 245us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:11:37.661255 (+ 593us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:11:37.661374 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:37.661405 (+ 31us) server_negotiation.cc:299] Negotiation successful
0806 21:11:37.661470 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 21:11:37.673918 20468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:37.670273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:51251 (local address 127.16.178.2:39749)
0806 21:11:37.670507 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:37.670511 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:37.670584 (+ 73us) server_negotiation.cc:407] Connection header received
0806 21:11:37.670724 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:37.670727 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:37.670772 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:37.670833 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:11:37.671100 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.671617 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.672887 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.673105 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.673181 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:37.673535 (+ 354us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:11:37.673606 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:11:37.673654 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:37.673747 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 21:11:37.673792 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147}
I20260806 21:11:37.674005 20523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:37.670134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.2:39749 (local address 127.16.178.1:51251)
0806 21:11:37.670500 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:37.670516 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:37.670633 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:37.670855 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:37.670858 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:37.670865 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 21:11:37.671013 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:37.671020 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.671712 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.671715 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:37.672767 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:37.672780 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.673209 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.673213 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:37.673311 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:37.673375 (+ 64us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 21:11:37.673671 (+ 296us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 21:11:37.673676 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 21:11:37.673734 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 21:11:37.673827 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":131,"threads_started":1}
I20260806 21:11:37.682873 20374 raft_consensus.cc:1261] T 8b486c5910c84d68a3df80962feba997 P 947641bfc7f24bef9d652d2bded18339 [term 1 FOLLOWER]: Refusing update from remote peer b832bfe3f9af4b40bb1a84d3d2669c01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:11:37.683130 20168 raft_consensus.cc:1261] T 8b486c5910c84d68a3df80962feba997 P 7bceff25285a4b1a818daf657bb0b467 [term 1 FOLLOWER]: Refusing update from remote peer b832bfe3f9af4b40bb1a84d3d2669c01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:11:37.683465 20486 consensus_queue.cc:1035] T 8b486c5910c84d68a3df80962feba997 P b832bfe3f9af4b40bb1a84d3d2669c01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:11:37.683743 20482 consensus_queue.cc:1035] T 8b486c5910c84d68a3df80962feba997 P b832bfe3f9af4b40bb1a84d3d2669c01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:11:37.712607 20465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:37.709141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:41151 (local address 127.16.178.1:35443)
0806 21:11:37.709282 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:37.709287 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:37.709304 (+ 17us) server_negotiation.cc:407] Connection header received
0806 21:11:37.709355 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:37.709358 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:37.709412 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:37.709481 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:11:37.709668 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.710480 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.711425 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.711734 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.711843 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:37.712186 (+ 343us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:11:37.712279 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:11:37.712336 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:37.712438 (+ 102us) server_negotiation.cc:299] Negotiation successful
0806 21:11:37.712494 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 21:11:37.712682 20530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:37.708795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:35443 (local address 127.16.178.2:41151)
0806 21:11:37.709151 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:37.709167 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:37.709268 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:37.709490 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:37.709492 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:37.709497 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 21:11:37.709588 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:37.709592 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.710599 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.710603 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:37.711301 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:37.711313 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:37.711858 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:37.711862 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:37.711952 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:37.712005 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 21:11:37.712356 (+ 351us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 21:11:37.712362 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 21:11:37.712430 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 21:11:37.712516 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":119,"threads_started":1}
I20260806 21:11:37.743449 19725 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54206:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:11:37.743662 19725 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54206:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:11:37.746491 19725 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a6079d59bd5d4b769d0ff72bdbd21121: Sending DeleteTablet for 3 replicas of tablet 8b486c5910c84d68a3df80962feba997
I20260806 21:11:37.747416 20342 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b486c5910c84d68a3df80962feba997 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:11:37 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:53238
I20260806 21:11:37.747586 19928 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b486c5910c84d68a3df80962feba997 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:11:37 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:42664
I20260806 21:11:37.748162 20536 tablet_replica.cc:330] T 8b486c5910c84d68a3df80962feba997 P 947641bfc7f24bef9d652d2bded18339: stopping tablet replica
I20260806 21:11:37.748543 20536 raft_consensus.cc:2229] T 8b486c5910c84d68a3df80962feba997 P 947641bfc7f24bef9d652d2bded18339 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:11:37.748840 20536 raft_consensus.cc:2258] T 8b486c5910c84d68a3df80962feba997 P 947641bfc7f24bef9d652d2bded18339 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:11:37.748883 20134 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b486c5910c84d68a3df80962feba997 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:11:37 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:40140
I20260806 21:11:37.749770 19725 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54206:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:11:37.750197 19725 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:11:37.750783 20536 ts_tablet_manager.cc:1905] T 8b486c5910c84d68a3df80962feba997 P 947641bfc7f24bef9d652d2bded18339: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:11:37.751703 20537 tablet_replica.cc:330] T 8b486c5910c84d68a3df80962feba997 P b832bfe3f9af4b40bb1a84d3d2669c01: stopping tablet replica
I20260806 21:11:37.751987 20537 raft_consensus.cc:2229] T 8b486c5910c84d68a3df80962feba997 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 1 LEADER]: Raft consensus shutting down.
I20260806 21:11:37.752347 20537 raft_consensus.cc:2258] T 8b486c5910c84d68a3df80962feba997 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:11:37.755743 20538 tablet_replica.cc:330] T 8b486c5910c84d68a3df80962feba997 P 7bceff25285a4b1a818daf657bb0b467: stopping tablet replica
I20260806 21:11:37.756031 20538 raft_consensus.cc:2229] T 8b486c5910c84d68a3df80962feba997 P 7bceff25285a4b1a818daf657bb0b467 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:11:37.756213 20538 raft_consensus.cc:2258] T 8b486c5910c84d68a3df80962feba997 P 7bceff25285a4b1a818daf657bb0b467 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:11:37.756376 19928 tablet_service.cc:1467] Processing CreateTablet for tablet 479826ce4f9d4b8da49befb9c17ae311 (DEFAULT_TABLE table=test-table [id=7c2b98c4851b4fe8950d7a3f4a1fd4e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:11:37.756567 20537 ts_tablet_manager.cc:1905] T 8b486c5910c84d68a3df80962feba997 P b832bfe3f9af4b40bb1a84d3d2669c01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:11:37.756661 19928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 479826ce4f9d4b8da49befb9c17ae311. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:37.758090 20538 ts_tablet_manager.cc:1905] T 8b486c5910c84d68a3df80962feba997 P 7bceff25285a4b1a818daf657bb0b467: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:11:37.759285 20134 tablet_service.cc:1467] Processing CreateTablet for tablet 479826ce4f9d4b8da49befb9c17ae311 (DEFAULT_TABLE table=test-table [id=7c2b98c4851b4fe8950d7a3f4a1fd4e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:11:37.759552 20134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 479826ce4f9d4b8da49befb9c17ae311. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:37.759763 20537 ts_tablet_manager.cc:1918] T 8b486c5910c84d68a3df80962feba997 P b832bfe3f9af4b40bb1a84d3d2669c01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 21:11:37.759848 20537 log.cc:1196] T 8b486c5910c84d68a3df80962feba997 P b832bfe3f9af4b40bb1a84d3d2669c01: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-0/wal/wals/8b486c5910c84d68a3df80962feba997
I20260806 21:11:37.760182 20537 ts_tablet_manager.cc:1939] T 8b486c5910c84d68a3df80962feba997 P b832bfe3f9af4b40bb1a84d3d2669c01: Deleting consensus metadata
I20260806 21:11:37.760392 20342 tablet_service.cc:1467] Processing CreateTablet for tablet 479826ce4f9d4b8da49befb9c17ae311 (DEFAULT_TABLE table=test-table [id=7c2b98c4851b4fe8950d7a3f4a1fd4e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:11:37.760670 20472 tablet_bootstrap.cc:492] T 479826ce4f9d4b8da49befb9c17ae311 P b832bfe3f9af4b40bb1a84d3d2669c01: Bootstrap starting.
I20260806 21:11:37.761616 20472 tablet_bootstrap.cc:654] T 479826ce4f9d4b8da49befb9c17ae311 P b832bfe3f9af4b40bb1a84d3d2669c01: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:37.762333 20538 ts_tablet_manager.cc:1918] T 8b486c5910c84d68a3df80962feba997 P 7bceff25285a4b1a818daf657bb0b467: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 21:11:37.762425 20538 log.cc:1196] T 8b486c5910c84d68a3df80962feba997 P 7bceff25285a4b1a818daf657bb0b467: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-1/wal/wals/8b486c5910c84d68a3df80962feba997
I20260806 21:11:37.762732 20538 ts_tablet_manager.cc:1939] T 8b486c5910c84d68a3df80962feba997 P 7bceff25285a4b1a818daf657bb0b467: Deleting consensus metadata
I20260806 21:11:37.762919 19704 catalog_manager.cc:4841] TS b832bfe3f9af4b40bb1a84d3d2669c01 (127.16.178.1:35443): tablet 8b486c5910c84d68a3df80962feba997 (table test-table [id=cdebba34b80c436e970ffb37249924c8]) successfully deleted
I20260806 21:11:37.765084 20473 tablet_bootstrap.cc:492] T 479826ce4f9d4b8da49befb9c17ae311 P 7bceff25285a4b1a818daf657bb0b467: Bootstrap starting.
I20260806 21:11:37.766099 20473 tablet_bootstrap.cc:654] T 479826ce4f9d4b8da49befb9c17ae311 P 7bceff25285a4b1a818daf657bb0b467: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:37.766952 19704 catalog_manager.cc:4841] TS 7bceff25285a4b1a818daf657bb0b467 (127.16.178.2:39749): tablet 8b486c5910c84d68a3df80962feba997 (table test-table [id=cdebba34b80c436e970ffb37249924c8]) successfully deleted
I20260806 21:11:37.767333 20473 tablet_bootstrap.cc:492] T 479826ce4f9d4b8da49befb9c17ae311 P 7bceff25285a4b1a818daf657bb0b467: No bootstrap required, opened a new log
I20260806 21:11:37.767416 20473 ts_tablet_manager.cc:1397] T 479826ce4f9d4b8da49befb9c17ae311 P 7bceff25285a4b1a818daf657bb0b467: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 21:11:37.767408 20342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 479826ce4f9d4b8da49befb9c17ae311. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:37.767834 20473 raft_consensus.cc:348] T 479826ce4f9d4b8da49befb9c17ae311 P 7bceff25285a4b1a818daf657bb0b467 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } }
I20260806 21:11:37.767952 20473 raft_consensus.cc:374] T 479826ce4f9d4b8da49befb9c17ae311 P 7bceff25285a4b1a818daf657bb0b467 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:37.767982 20473 raft_consensus.cc:729] T 479826ce4f9d4b8da49befb9c17ae311 P 7bceff25285a4b1a818daf657bb0b467 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bceff25285a4b1a818daf657bb0b467, State: Initialized, Role: FOLLOWER
I20260806 21:11:37.768133 20473 consensus_queue.cc:260] T 479826ce4f9d4b8da49befb9c17ae311 P 7bceff25285a4b1a818daf657bb0b467 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } }
I20260806 21:11:37.768875 20472 tablet_bootstrap.cc:492] T 479826ce4f9d4b8da49befb9c17ae311 P b832bfe3f9af4b40bb1a84d3d2669c01: No bootstrap required, opened a new log
I20260806 21:11:37.768967 20472 ts_tablet_manager.cc:1397] T 479826ce4f9d4b8da49befb9c17ae311 P b832bfe3f9af4b40bb1a84d3d2669c01: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260806 21:11:37.769414 20472 raft_consensus.cc:348] T 479826ce4f9d4b8da49befb9c17ae311 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } }
I20260806 21:11:37.769546 20472 raft_consensus.cc:374] T 479826ce4f9d4b8da49befb9c17ae311 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:37.769579 20472 raft_consensus.cc:729] T 479826ce4f9d4b8da49befb9c17ae311 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b832bfe3f9af4b40bb1a84d3d2669c01, State: Initialized, Role: FOLLOWER
I20260806 21:11:37.769701 20472 consensus_queue.cc:260] T 479826ce4f9d4b8da49befb9c17ae311 P b832bfe3f9af4b40bb1a84d3d2669c01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } }
I20260806 21:11:37.768397 20473 ts_tablet_manager.cc:1428] T 479826ce4f9d4b8da49befb9c17ae311 P 7bceff25285a4b1a818daf657bb0b467: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 21:11:37.769979 20472 ts_tablet_manager.cc:1428] T 479826ce4f9d4b8da49befb9c17ae311 P b832bfe3f9af4b40bb1a84d3d2669c01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 21:11:37.770124 20475 tablet_bootstrap.cc:492] T 479826ce4f9d4b8da49befb9c17ae311 P 947641bfc7f24bef9d652d2bded18339: Bootstrap starting.
I20260806 21:11:37.771121 20475 tablet_bootstrap.cc:654] T 479826ce4f9d4b8da49befb9c17ae311 P 947641bfc7f24bef9d652d2bded18339: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:37.772260 20475 tablet_bootstrap.cc:492] T 479826ce4f9d4b8da49befb9c17ae311 P 947641bfc7f24bef9d652d2bded18339: No bootstrap required, opened a new log
I20260806 21:11:37.772341 20475 ts_tablet_manager.cc:1397] T 479826ce4f9d4b8da49befb9c17ae311 P 947641bfc7f24bef9d652d2bded18339: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 21:11:37.772753 20475 raft_consensus.cc:348] T 479826ce4f9d4b8da49befb9c17ae311 P 947641bfc7f24bef9d652d2bded18339 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } }
I20260806 21:11:37.772867 20475 raft_consensus.cc:374] T 479826ce4f9d4b8da49befb9c17ae311 P 947641bfc7f24bef9d652d2bded18339 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:37.772904 20475 raft_consensus.cc:729] T 479826ce4f9d4b8da49befb9c17ae311 P 947641bfc7f24bef9d652d2bded18339 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 947641bfc7f24bef9d652d2bded18339, State: Initialized, Role: FOLLOWER
I20260806 21:11:37.773010 20475 consensus_queue.cc:260] T 479826ce4f9d4b8da49befb9c17ae311 P 947641bfc7f24bef9d652d2bded18339 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } }
I20260806 21:11:37.773232 20475 ts_tablet_manager.cc:1428] T 479826ce4f9d4b8da49befb9c17ae311 P 947641bfc7f24bef9d652d2bded18339: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 21:11:37.776467 20536 ts_tablet_manager.cc:1918] T 8b486c5910c84d68a3df80962feba997 P 947641bfc7f24bef9d652d2bded18339: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 21:11:37.776620 20536 log.cc:1196] T 8b486c5910c84d68a3df80962feba997 P 947641bfc7f24bef9d652d2bded18339: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-2/wal/wals/8b486c5910c84d68a3df80962feba997
I20260806 21:11:37.777009 20536 ts_tablet_manager.cc:1939] T 8b486c5910c84d68a3df80962feba997 P 947641bfc7f24bef9d652d2bded18339: Deleting consensus metadata
I20260806 21:11:37.778436 19704 catalog_manager.cc:4841] TS 947641bfc7f24bef9d652d2bded18339 (127.16.178.3:42075): tablet 8b486c5910c84d68a3df80962feba997 (table test-table [id=cdebba34b80c436e970ffb37249924c8]) successfully deleted
I20260806 21:11:37.828212 20527 raft_consensus.cc:482] T 479826ce4f9d4b8da49befb9c17ae311 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:11:37.828464 20527 raft_consensus.cc:504] T 479826ce4f9d4b8da49befb9c17ae311 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } }
I20260806 21:11:37.828939 20527 leader_election.cc:288] T 479826ce4f9d4b8da49befb9c17ae311 P b832bfe3f9af4b40bb1a84d3d2669c01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bceff25285a4b1a818daf657bb0b467 (127.16.178.2:39749), 947641bfc7f24bef9d652d2bded18339 (127.16.178.3:42075)
I20260806 21:11:37.829435 20168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "479826ce4f9d4b8da49befb9c17ae311" candidate_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bceff25285a4b1a818daf657bb0b467" is_pre_election: true
I20260806 21:11:37.829489 20374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "479826ce4f9d4b8da49befb9c17ae311" candidate_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "947641bfc7f24bef9d652d2bded18339" is_pre_election: true
I20260806 21:11:37.829638 20374 raft_consensus.cc:2454] T 479826ce4f9d4b8da49befb9c17ae311 P 947641bfc7f24bef9d652d2bded18339 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b832bfe3f9af4b40bb1a84d3d2669c01 in term 0.
I20260806 21:11:37.829619 20168 raft_consensus.cc:2454] T 479826ce4f9d4b8da49befb9c17ae311 P 7bceff25285a4b1a818daf657bb0b467 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b832bfe3f9af4b40bb1a84d3d2669c01 in term 0.
I20260806 21:11:37.829941 19857 leader_election.cc:302] T 479826ce4f9d4b8da49befb9c17ae311 P b832bfe3f9af4b40bb1a84d3d2669c01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bceff25285a4b1a818daf657bb0b467, b832bfe3f9af4b40bb1a84d3d2669c01; no voters:
I20260806 21:11:37.830106 20527 raft_consensus.cc:2781] T 479826ce4f9d4b8da49befb9c17ae311 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:11:37.830168 20527 raft_consensus.cc:482] T 479826ce4f9d4b8da49befb9c17ae311 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:11:37.830205 20527 raft_consensus.cc:3037] T 479826ce4f9d4b8da49befb9c17ae311 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:37.831223 20527 raft_consensus.cc:504] T 479826ce4f9d4b8da49befb9c17ae311 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } }
I20260806 21:11:37.831584 20527 leader_election.cc:288] T 479826ce4f9d4b8da49befb9c17ae311 P b832bfe3f9af4b40bb1a84d3d2669c01 [CANDIDATE]: Term 1 election: Requested vote from peers 7bceff25285a4b1a818daf657bb0b467 (127.16.178.2:39749), 947641bfc7f24bef9d652d2bded18339 (127.16.178.3:42075)
I20260806 21:11:37.831981 20374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "479826ce4f9d4b8da49befb9c17ae311" candidate_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "947641bfc7f24bef9d652d2bded18339"
I20260806 21:11:37.831971 20168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "479826ce4f9d4b8da49befb9c17ae311" candidate_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bceff25285a4b1a818daf657bb0b467"
I20260806 21:11:37.832136 20374 raft_consensus.cc:3037] T 479826ce4f9d4b8da49befb9c17ae311 P 947641bfc7f24bef9d652d2bded18339 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:37.832134 20168 raft_consensus.cc:3037] T 479826ce4f9d4b8da49befb9c17ae311 P 7bceff25285a4b1a818daf657bb0b467 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:37.833263 20374 raft_consensus.cc:2454] T 479826ce4f9d4b8da49befb9c17ae311 P 947641bfc7f24bef9d652d2bded18339 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b832bfe3f9af4b40bb1a84d3d2669c01 in term 1.
I20260806 21:11:37.833263 20168 raft_consensus.cc:2454] T 479826ce4f9d4b8da49befb9c17ae311 P 7bceff25285a4b1a818daf657bb0b467 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b832bfe3f9af4b40bb1a84d3d2669c01 in term 1.
I20260806 21:11:37.833580 19857 leader_election.cc:302] T 479826ce4f9d4b8da49befb9c17ae311 P b832bfe3f9af4b40bb1a84d3d2669c01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 947641bfc7f24bef9d652d2bded18339, b832bfe3f9af4b40bb1a84d3d2669c01; no voters:
I20260806 21:11:37.833765 20527 raft_consensus.cc:2781] T 479826ce4f9d4b8da49befb9c17ae311 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:11:37.833860 20527 raft_consensus.cc:686] T 479826ce4f9d4b8da49befb9c17ae311 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 1 LEADER]: Becoming Leader. State: Replica: b832bfe3f9af4b40bb1a84d3d2669c01, State: Running, Role: LEADER
I20260806 21:11:37.834003 20527 consensus_queue.cc:237] T 479826ce4f9d4b8da49befb9c17ae311 P b832bfe3f9af4b40bb1a84d3d2669c01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } }
I20260806 21:11:37.836012 19725 catalog_manager.cc:5495] T 479826ce4f9d4b8da49befb9c17ae311 P b832bfe3f9af4b40bb1a84d3d2669c01 reported cstate change: term changed from 0 to 1, leader changed from <none> to b832bfe3f9af4b40bb1a84d3d2669c01 (127.16.178.1). New cstate: current_term: 1 leader_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } health_report { overall_health: UNKNOWN } } }
I20260806 21:11:37.875084 20374 raft_consensus.cc:1261] T 479826ce4f9d4b8da49befb9c17ae311 P 947641bfc7f24bef9d652d2bded18339 [term 1 FOLLOWER]: Refusing update from remote peer b832bfe3f9af4b40bb1a84d3d2669c01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:11:37.875084 20168 raft_consensus.cc:1261] T 479826ce4f9d4b8da49befb9c17ae311 P 7bceff25285a4b1a818daf657bb0b467 [term 1 FOLLOWER]: Refusing update from remote peer b832bfe3f9af4b40bb1a84d3d2669c01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:11:37.875536 20482 consensus_queue.cc:1035] T 479826ce4f9d4b8da49befb9c17ae311 P b832bfe3f9af4b40bb1a84d3d2669c01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:11:37.875733 20527 consensus_queue.cc:1035] T 479826ce4f9d4b8da49befb9c17ae311 P b832bfe3f9af4b40bb1a84d3d2669c01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:11:37.879890 20510 mvcc.cc:205] Tried to move back new op lower bound from 7315663658492510208 to 7315663658329264128. Current Snapshot: MvccSnapshot[applied={T|T < 7315663658492510208}]
W20260806 21:11:37.893786 20512 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 21:11:39.453004 19725 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54206:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:11:39.453186 19725 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54206:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:11:39.455753 19725 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a6079d59bd5d4b769d0ff72bdbd21121: Sending DeleteTablet for 3 replicas of tablet 479826ce4f9d4b8da49befb9c17ae311
I20260806 21:11:39.456373 19928 tablet_service.cc:1514] Processing DeleteTablet for tablet 479826ce4f9d4b8da49befb9c17ae311 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:11:39 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:42664
I20260806 21:11:39.456501 20134 tablet_service.cc:1514] Processing DeleteTablet for tablet 479826ce4f9d4b8da49befb9c17ae311 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:11:39 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:40140
I20260806 21:11:39.456786 20342 tablet_service.cc:1514] Processing DeleteTablet for tablet 479826ce4f9d4b8da49befb9c17ae311 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:11:39 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:53238
I20260806 21:11:39.457106 20584 tablet_replica.cc:330] T 479826ce4f9d4b8da49befb9c17ae311 P b832bfe3f9af4b40bb1a84d3d2669c01: stopping tablet replica
I20260806 21:11:39.457288 20584 raft_consensus.cc:2229] T 479826ce4f9d4b8da49befb9c17ae311 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 1 LEADER]: Raft consensus shutting down.
I20260806 21:11:39.457507 20586 tablet_replica.cc:330] T 479826ce4f9d4b8da49befb9c17ae311 P 947641bfc7f24bef9d652d2bded18339: stopping tablet replica
I20260806 21:11:39.457545 20584 raft_consensus.cc:2258] T 479826ce4f9d4b8da49befb9c17ae311 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:11:39.457705 20586 raft_consensus.cc:2229] T 479826ce4f9d4b8da49befb9c17ae311 P 947641bfc7f24bef9d652d2bded18339 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:11:39.457855 20586 raft_consensus.cc:2258] T 479826ce4f9d4b8da49befb9c17ae311 P 947641bfc7f24bef9d652d2bded18339 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:11:39.458240 20584 ts_tablet_manager.cc:1905] T 479826ce4f9d4b8da49befb9c17ae311 P b832bfe3f9af4b40bb1a84d3d2669c01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:11:39.458451 20586 ts_tablet_manager.cc:1905] T 479826ce4f9d4b8da49befb9c17ae311 P 947641bfc7f24bef9d652d2bded18339: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:11:39.459288 20585 tablet_replica.cc:330] T 479826ce4f9d4b8da49befb9c17ae311 P 7bceff25285a4b1a818daf657bb0b467: stopping tablet replica
I20260806 21:11:39.459475 20585 raft_consensus.cc:2229] T 479826ce4f9d4b8da49befb9c17ae311 P 7bceff25285a4b1a818daf657bb0b467 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:11:39.459656 20585 raft_consensus.cc:2258] T 479826ce4f9d4b8da49befb9c17ae311 P 7bceff25285a4b1a818daf657bb0b467 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:11:39.460420 20585 ts_tablet_manager.cc:1905] T 479826ce4f9d4b8da49befb9c17ae311 P 7bceff25285a4b1a818daf657bb0b467: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:11:39.461187 20586 ts_tablet_manager.cc:1918] T 479826ce4f9d4b8da49befb9c17ae311 P 947641bfc7f24bef9d652d2bded18339: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 21:11:39.461256 20584 ts_tablet_manager.cc:1918] T 479826ce4f9d4b8da49befb9c17ae311 P b832bfe3f9af4b40bb1a84d3d2669c01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 21:11:39.461315 20586 log.cc:1196] T 479826ce4f9d4b8da49befb9c17ae311 P 947641bfc7f24bef9d652d2bded18339: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-2/wal/wals/479826ce4f9d4b8da49befb9c17ae311
I20260806 21:11:39.461426 20584 log.cc:1196] T 479826ce4f9d4b8da49befb9c17ae311 P b832bfe3f9af4b40bb1a84d3d2669c01: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-0/wal/wals/479826ce4f9d4b8da49befb9c17ae311
I20260806 21:11:39.461623 20586 ts_tablet_manager.cc:1939] T 479826ce4f9d4b8da49befb9c17ae311 P 947641bfc7f24bef9d652d2bded18339: Deleting consensus metadata
I20260806 21:11:39.461764 20584 ts_tablet_manager.cc:1939] T 479826ce4f9d4b8da49befb9c17ae311 P b832bfe3f9af4b40bb1a84d3d2669c01: Deleting consensus metadata
I20260806 21:11:39.462510 19704 catalog_manager.cc:4841] TS 947641bfc7f24bef9d652d2bded18339 (127.16.178.3:42075): tablet 479826ce4f9d4b8da49befb9c17ae311 (table test-table [id=7c2b98c4851b4fe8950d7a3f4a1fd4e0]) successfully deleted
I20260806 21:11:39.462698 19704 catalog_manager.cc:4841] TS b832bfe3f9af4b40bb1a84d3d2669c01 (127.16.178.1:35443): tablet 479826ce4f9d4b8da49befb9c17ae311 (table test-table [id=7c2b98c4851b4fe8950d7a3f4a1fd4e0]) successfully deleted
I20260806 21:11:39.462954 20585 ts_tablet_manager.cc:1918] T 479826ce4f9d4b8da49befb9c17ae311 P 7bceff25285a4b1a818daf657bb0b467: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 21:11:39.463020 20585 log.cc:1196] T 479826ce4f9d4b8da49befb9c17ae311 P 7bceff25285a4b1a818daf657bb0b467: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-1/wal/wals/479826ce4f9d4b8da49befb9c17ae311
I20260806 21:11:39.463196 20585 ts_tablet_manager.cc:1939] T 479826ce4f9d4b8da49befb9c17ae311 P 7bceff25285a4b1a818daf657bb0b467: Deleting consensus metadata
I20260806 21:11:39.464061 19704 catalog_manager.cc:4841] TS 7bceff25285a4b1a818daf657bb0b467 (127.16.178.2:39749): tablet 479826ce4f9d4b8da49befb9c17ae311 (table test-table [id=7c2b98c4851b4fe8950d7a3f4a1fd4e0]) successfully deleted
Aug 06 21:11:39 dist-test-slave-lgr5 krb5kdc[19661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050699, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 21:11:39 dist-test-slave-lgr5 krb5kdc[19661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050699, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:11:39.490272 20600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:39.480939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54224 (local address 127.16.178.62:45481)
0806 21:11:39.481235 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:39.481242 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:39.481261 (+ 19us) server_negotiation.cc:407] Connection header received
0806 21:11:39.481396 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:39.481399 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:39.481474 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:39.481570 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:39.482479 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:39.483064 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:39.483960 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:39.484285 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:39.484403 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:39.486930 (+ 2527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:39.486934 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:39.486936 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:39.486966 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:39.488437 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:39.488915 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:39.488918 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:39.488919 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:39.489004 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:39.489264 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:39.489267 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:39.489269 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:39.489513 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:39.489665 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:39.489963 (+ 298us) server_negotiation.cc:299] Negotiation successful
0806 21:11:39.490094 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":102,"threads_started":1}
I20260806 21:11:39.493330 19725 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54224:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:11:39.493824 19725 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:11:39.499200 20342 tablet_service.cc:1467] Processing CreateTablet for tablet 805d9e7aa68b4fc5bed0d5817a5e2a96 (DEFAULT_TABLE table=test-table [id=1fd1e114739a44858c1543011c31d66a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:11:39.499200 19928 tablet_service.cc:1467] Processing CreateTablet for tablet 805d9e7aa68b4fc5bed0d5817a5e2a96 (DEFAULT_TABLE table=test-table [id=1fd1e114739a44858c1543011c31d66a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:11:39.499245 20134 tablet_service.cc:1467] Processing CreateTablet for tablet 805d9e7aa68b4fc5bed0d5817a5e2a96 (DEFAULT_TABLE table=test-table [id=1fd1e114739a44858c1543011c31d66a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:11:39.499557 20342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 805d9e7aa68b4fc5bed0d5817a5e2a96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:39.499557 19928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 805d9e7aa68b4fc5bed0d5817a5e2a96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:39.499589 20134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 805d9e7aa68b4fc5bed0d5817a5e2a96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:39.501827 20601 tablet_bootstrap.cc:492] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 7bceff25285a4b1a818daf657bb0b467: Bootstrap starting.
I20260806 21:11:39.502314 20603 tablet_bootstrap.cc:492] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 947641bfc7f24bef9d652d2bded18339: Bootstrap starting.
I20260806 21:11:39.502759 20605 tablet_bootstrap.cc:492] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P b832bfe3f9af4b40bb1a84d3d2669c01: Bootstrap starting.
I20260806 21:11:39.502789 20601 tablet_bootstrap.cc:654] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 7bceff25285a4b1a818daf657bb0b467: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:39.503252 20603 tablet_bootstrap.cc:654] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 947641bfc7f24bef9d652d2bded18339: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:39.503782 20605 tablet_bootstrap.cc:654] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P b832bfe3f9af4b40bb1a84d3d2669c01: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:39.503993 20601 tablet_bootstrap.cc:492] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 7bceff25285a4b1a818daf657bb0b467: No bootstrap required, opened a new log
I20260806 21:11:39.504124 20601 ts_tablet_manager.cc:1397] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 7bceff25285a4b1a818daf657bb0b467: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 21:11:39.504601 20603 tablet_bootstrap.cc:492] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 947641bfc7f24bef9d652d2bded18339: No bootstrap required, opened a new log
I20260806 21:11:39.504526 20601 raft_consensus.cc:348] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 7bceff25285a4b1a818daf657bb0b467 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } }
I20260806 21:11:39.504647 20601 raft_consensus.cc:374] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 7bceff25285a4b1a818daf657bb0b467 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:39.504681 20601 raft_consensus.cc:729] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 7bceff25285a4b1a818daf657bb0b467 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bceff25285a4b1a818daf657bb0b467, State: Initialized, Role: FOLLOWER
I20260806 21:11:39.504683 20603 ts_tablet_manager.cc:1397] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 947641bfc7f24bef9d652d2bded18339: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 21:11:39.504855 20601 consensus_queue.cc:260] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 7bceff25285a4b1a818daf657bb0b467 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } }
I20260806 21:11:39.505146 20603 raft_consensus.cc:348] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 947641bfc7f24bef9d652d2bded18339 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } }
I20260806 21:11:39.505298 20603 raft_consensus.cc:374] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 947641bfc7f24bef9d652d2bded18339 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:39.505366 20603 raft_consensus.cc:729] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 947641bfc7f24bef9d652d2bded18339 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 947641bfc7f24bef9d652d2bded18339, State: Initialized, Role: FOLLOWER
I20260806 21:11:39.505532 20603 consensus_queue.cc:260] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 947641bfc7f24bef9d652d2bded18339 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } }
I20260806 21:11:39.505620 20601 ts_tablet_manager.cc:1428] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 7bceff25285a4b1a818daf657bb0b467: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 21:11:39.506024 20603 ts_tablet_manager.cc:1428] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 947641bfc7f24bef9d652d2bded18339: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 21:11:39.506084 20605 tablet_bootstrap.cc:492] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P b832bfe3f9af4b40bb1a84d3d2669c01: No bootstrap required, opened a new log
I20260806 21:11:39.506388 20605 ts_tablet_manager.cc:1397] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P b832bfe3f9af4b40bb1a84d3d2669c01: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 21:11:39.506840 20605 raft_consensus.cc:348] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } }
I20260806 21:11:39.506999 20605 raft_consensus.cc:374] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:39.507050 20605 raft_consensus.cc:729] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b832bfe3f9af4b40bb1a84d3d2669c01, State: Initialized, Role: FOLLOWER
I20260806 21:11:39.507210 20605 consensus_queue.cc:260] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P b832bfe3f9af4b40bb1a84d3d2669c01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } }
I20260806 21:11:39.507509 20605 ts_tablet_manager.cc:1428] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P b832bfe3f9af4b40bb1a84d3d2669c01: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 21:11:39.639725 20551 raft_consensus.cc:482] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 7bceff25285a4b1a818daf657bb0b467 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:11:39.639945 20551 raft_consensus.cc:504] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 7bceff25285a4b1a818daf657bb0b467 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } }
I20260806 21:11:39.640466 20551 leader_election.cc:288] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 7bceff25285a4b1a818daf657bb0b467 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b832bfe3f9af4b40bb1a84d3d2669c01 (127.16.178.1:35443), 947641bfc7f24bef9d652d2bded18339 (127.16.178.3:42075)
I20260806 21:11:39.640987 19955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "805d9e7aa68b4fc5bed0d5817a5e2a96" candidate_uuid: "7bceff25285a4b1a818daf657bb0b467" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" is_pre_election: true
I20260806 21:11:39.641049 20374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "805d9e7aa68b4fc5bed0d5817a5e2a96" candidate_uuid: "7bceff25285a4b1a818daf657bb0b467" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "947641bfc7f24bef9d652d2bded18339" is_pre_election: true
I20260806 21:11:39.641155 19955 raft_consensus.cc:2454] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bceff25285a4b1a818daf657bb0b467 in term 0.
I20260806 21:11:39.641237 20374 raft_consensus.cc:2454] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 947641bfc7f24bef9d652d2bded18339 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bceff25285a4b1a818daf657bb0b467 in term 0.
I20260806 21:11:39.641582 20066 leader_election.cc:302] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 7bceff25285a4b1a818daf657bb0b467 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bceff25285a4b1a818daf657bb0b467, b832bfe3f9af4b40bb1a84d3d2669c01; no voters:
I20260806 21:11:39.641808 20551 raft_consensus.cc:2781] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 7bceff25285a4b1a818daf657bb0b467 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:11:39.641878 20551 raft_consensus.cc:482] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 7bceff25285a4b1a818daf657bb0b467 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:11:39.641923 20551 raft_consensus.cc:3037] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 7bceff25285a4b1a818daf657bb0b467 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:39.643097 20551 raft_consensus.cc:504] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 7bceff25285a4b1a818daf657bb0b467 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } }
I20260806 21:11:39.643489 20551 leader_election.cc:288] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 7bceff25285a4b1a818daf657bb0b467 [CANDIDATE]: Term 1 election: Requested vote from peers b832bfe3f9af4b40bb1a84d3d2669c01 (127.16.178.1:35443), 947641bfc7f24bef9d652d2bded18339 (127.16.178.3:42075)
I20260806 21:11:39.644006 19955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "805d9e7aa68b4fc5bed0d5817a5e2a96" candidate_uuid: "7bceff25285a4b1a818daf657bb0b467" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01"
I20260806 21:11:39.644068 20374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "805d9e7aa68b4fc5bed0d5817a5e2a96" candidate_uuid: "7bceff25285a4b1a818daf657bb0b467" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "947641bfc7f24bef9d652d2bded18339"
I20260806 21:11:39.644200 19955 raft_consensus.cc:3037] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:39.644232 20374 raft_consensus.cc:3037] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 947641bfc7f24bef9d652d2bded18339 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:39.645195 20374 raft_consensus.cc:2454] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 947641bfc7f24bef9d652d2bded18339 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bceff25285a4b1a818daf657bb0b467 in term 1.
I20260806 21:11:39.645262 19955 raft_consensus.cc:2454] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bceff25285a4b1a818daf657bb0b467 in term 1.
I20260806 21:11:39.645529 20066 leader_election.cc:302] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 7bceff25285a4b1a818daf657bb0b467 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bceff25285a4b1a818daf657bb0b467, 947641bfc7f24bef9d652d2bded18339; no voters:
I20260806 21:11:39.645697 20551 raft_consensus.cc:2781] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 7bceff25285a4b1a818daf657bb0b467 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:11:39.645794 20551 raft_consensus.cc:686] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 7bceff25285a4b1a818daf657bb0b467 [term 1 LEADER]: Becoming Leader. State: Replica: 7bceff25285a4b1a818daf657bb0b467, State: Running, Role: LEADER
I20260806 21:11:39.645931 20551 consensus_queue.cc:237] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 7bceff25285a4b1a818daf657bb0b467 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } }
I20260806 21:11:39.647816 19713 catalog_manager.cc:5495] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 7bceff25285a4b1a818daf657bb0b467 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7bceff25285a4b1a818daf657bb0b467 (127.16.178.2). New cstate: current_term: 1 leader_uuid: "7bceff25285a4b1a818daf657bb0b467" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } health_report { overall_health: UNKNOWN } } }
I20260806 21:11:39.669147 20615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:39.665973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50592 (local address 127.16.178.2:39749)
0806 21:11:39.666244 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:39.666251 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:39.666261 (+ 10us) server_negotiation.cc:407] Connection header received
0806 21:11:39.666298 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:39.666300 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:39.666340 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:39.666411 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:11:39.666695 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:39.667225 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:39.668066 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:39.668331 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:39.668421 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:39.668677 (+ 256us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:11:39.668751 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:11:39.668809 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:39.668933 (+ 124us) server_negotiation.cc:299] Negotiation successful
0806 21:11:39.668991 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":99,"threads_started":1}
I20260806 21:11:39.678036 20619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:39.675097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.2:39749 (local address 127.16.178.2:51645)
0806 21:11:39.675384 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:39.675397 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:39.675486 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:39.675670 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:39.675671 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:39.675677 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 21:11:39.675760 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:39.675764 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:39.676407 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:39.676409 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:39.677012 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:39.677021 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:39.677409 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:39.677412 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:39.677487 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:39.677514 (+ 27us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 21:11:39.677737 (+ 223us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 21:11:39.677743 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 21:11:39.677792 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 21:11:39.677858 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":86,"threads_started":1}
I20260806 21:11:39.678043 20615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:39.675171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:51645 (local address 127.16.178.2:39749)
0806 21:11:39.675280 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:39.675283 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:39.675476 (+ 193us) server_negotiation.cc:407] Connection header received
0806 21:11:39.675556 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:39.675558 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:39.675599 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:39.675659 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:11:39.675831 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:39.676322 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:39.677103 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:39.677300 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:39.677369 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:39.677609 (+ 240us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:11:39.677672 (+ 63us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:11:39.677714 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:39.677804 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 21:11:39.677848 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":41}
I20260806 21:11:39.684121 19955 raft_consensus.cc:1261] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 1 FOLLOWER]: Refusing update from remote peer 7bceff25285a4b1a818daf657bb0b467: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:11:39.684154 20374 raft_consensus.cc:1261] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 947641bfc7f24bef9d652d2bded18339 [term 1 FOLLOWER]: Refusing update from remote peer 7bceff25285a4b1a818daf657bb0b467: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:11:39.684502 20563 consensus_queue.cc:1035] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 7bceff25285a4b1a818daf657bb0b467 [LEADER]: Connected to new peer: Peer: permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:11:39.684664 20554 consensus_queue.cc:1035] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 7bceff25285a4b1a818daf657bb0b467 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:11:39.718480 19725 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:11:39.718652 19725 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:11:39.721274 19725 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a6079d59bd5d4b769d0ff72bdbd21121: Sending DeleteTablet for 3 replicas of tablet 805d9e7aa68b4fc5bed0d5817a5e2a96
I20260806 21:11:39.722126 19928 tablet_service.cc:1514] Processing DeleteTablet for tablet 805d9e7aa68b4fc5bed0d5817a5e2a96 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:11:39 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:42664
I20260806 21:11:39.722364 20584 tablet_replica.cc:330] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P b832bfe3f9af4b40bb1a84d3d2669c01: stopping tablet replica
I20260806 21:11:39.722414 20342 tablet_service.cc:1514] Processing DeleteTablet for tablet 805d9e7aa68b4fc5bed0d5817a5e2a96 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:11:39 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:53238
I20260806 21:11:39.722500 20584 raft_consensus.cc:2229] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:11:39.722638 20584 raft_consensus.cc:2258] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:11:39.722838 20586 tablet_replica.cc:330] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 947641bfc7f24bef9d652d2bded18339: stopping tablet replica
I20260806 21:11:39.722959 20586 raft_consensus.cc:2229] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 947641bfc7f24bef9d652d2bded18339 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:11:39.723067 20586 raft_consensus.cc:2258] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 947641bfc7f24bef9d652d2bded18339 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:11:39.723054 19725 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54224:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:11:39.723515 19725 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:11:39.724000 20586 ts_tablet_manager.cc:1905] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 947641bfc7f24bef9d652d2bded18339: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:11:39.724038 20584 ts_tablet_manager.cc:1905] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P b832bfe3f9af4b40bb1a84d3d2669c01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:11:39.726281 20586 ts_tablet_manager.cc:1918] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 947641bfc7f24bef9d652d2bded18339: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 21:11:39.726367 20586 log.cc:1196] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 947641bfc7f24bef9d652d2bded18339: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-2/wal/wals/805d9e7aa68b4fc5bed0d5817a5e2a96
I20260806 21:11:39.726620 20586 ts_tablet_manager.cc:1939] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 947641bfc7f24bef9d652d2bded18339: Deleting consensus metadata
I20260806 21:11:39.727057 20584 ts_tablet_manager.cc:1918] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P b832bfe3f9af4b40bb1a84d3d2669c01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 21:11:39.727149 20584 log.cc:1196] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P b832bfe3f9af4b40bb1a84d3d2669c01: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-0/wal/wals/805d9e7aa68b4fc5bed0d5817a5e2a96
I20260806 21:11:39.728133 19704 catalog_manager.cc:4841] TS 947641bfc7f24bef9d652d2bded18339 (127.16.178.3:42075): tablet 805d9e7aa68b4fc5bed0d5817a5e2a96 (table test-table [id=1fd1e114739a44858c1543011c31d66a]) successfully deleted
I20260806 21:11:39.728274 20134 tablet_service.cc:1514] Processing DeleteTablet for tablet 805d9e7aa68b4fc5bed0d5817a5e2a96 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:11:39 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:40140
I20260806 21:11:39.728510 20585 tablet_replica.cc:330] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 7bceff25285a4b1a818daf657bb0b467: stopping tablet replica
I20260806 21:11:39.728637 20585 raft_consensus.cc:2229] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 7bceff25285a4b1a818daf657bb0b467 [term 1 LEADER]: Raft consensus shutting down.
I20260806 21:11:39.729013 20585 raft_consensus.cc:2258] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 7bceff25285a4b1a818daf657bb0b467 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:11:39.729838 20584 ts_tablet_manager.cc:1939] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P b832bfe3f9af4b40bb1a84d3d2669c01: Deleting consensus metadata
I20260806 21:11:39.729955 20585 ts_tablet_manager.cc:1905] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 7bceff25285a4b1a818daf657bb0b467: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:11:39.730820 19704 catalog_manager.cc:4841] TS b832bfe3f9af4b40bb1a84d3d2669c01 (127.16.178.1:35443): tablet 805d9e7aa68b4fc5bed0d5817a5e2a96 (table test-table [id=1fd1e114739a44858c1543011c31d66a]) successfully deleted
I20260806 21:11:39.732617 20585 ts_tablet_manager.cc:1918] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 7bceff25285a4b1a818daf657bb0b467: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 21:11:39.732695 20585 log.cc:1196] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 7bceff25285a4b1a818daf657bb0b467: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-1/wal/wals/805d9e7aa68b4fc5bed0d5817a5e2a96
I20260806 21:11:39.732946 20585 ts_tablet_manager.cc:1939] T 805d9e7aa68b4fc5bed0d5817a5e2a96 P 7bceff25285a4b1a818daf657bb0b467: Deleting consensus metadata
I20260806 21:11:39.733745 20342 tablet_service.cc:1467] Processing CreateTablet for tablet 40b234b98a6445af8f65d0133f86b751 (DEFAULT_TABLE table=test-table [id=72f901e2e1934bd391407b4afcf4c38a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:11:39.734050 20342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40b234b98a6445af8f65d0133f86b751. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:39.734468 20134 tablet_service.cc:1467] Processing CreateTablet for tablet 40b234b98a6445af8f65d0133f86b751 (DEFAULT_TABLE table=test-table [id=72f901e2e1934bd391407b4afcf4c38a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:11:39.734476 19928 tablet_service.cc:1467] Processing CreateTablet for tablet 40b234b98a6445af8f65d0133f86b751 (DEFAULT_TABLE table=test-table [id=72f901e2e1934bd391407b4afcf4c38a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:11:39.734714 19928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40b234b98a6445af8f65d0133f86b751. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:39.734715 20134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40b234b98a6445af8f65d0133f86b751. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:39.736807 19704 catalog_manager.cc:4841] TS 7bceff25285a4b1a818daf657bb0b467 (127.16.178.2:39749): tablet 805d9e7aa68b4fc5bed0d5817a5e2a96 (table test-table [id=1fd1e114739a44858c1543011c31d66a]) successfully deleted
I20260806 21:11:39.737057 20601 tablet_bootstrap.cc:492] T 40b234b98a6445af8f65d0133f86b751 P 7bceff25285a4b1a818daf657bb0b467: Bootstrap starting.
I20260806 21:11:39.737134 20605 tablet_bootstrap.cc:492] T 40b234b98a6445af8f65d0133f86b751 P b832bfe3f9af4b40bb1a84d3d2669c01: Bootstrap starting.
I20260806 21:11:39.737345 20603 tablet_bootstrap.cc:492] T 40b234b98a6445af8f65d0133f86b751 P 947641bfc7f24bef9d652d2bded18339: Bootstrap starting.
I20260806 21:11:39.738039 20605 tablet_bootstrap.cc:654] T 40b234b98a6445af8f65d0133f86b751 P b832bfe3f9af4b40bb1a84d3d2669c01: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:39.738091 20601 tablet_bootstrap.cc:654] T 40b234b98a6445af8f65d0133f86b751 P 7bceff25285a4b1a818daf657bb0b467: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:39.738313 20603 tablet_bootstrap.cc:654] T 40b234b98a6445af8f65d0133f86b751 P 947641bfc7f24bef9d652d2bded18339: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:39.739912 20603 tablet_bootstrap.cc:492] T 40b234b98a6445af8f65d0133f86b751 P 947641bfc7f24bef9d652d2bded18339: No bootstrap required, opened a new log
I20260806 21:11:39.739912 20601 tablet_bootstrap.cc:492] T 40b234b98a6445af8f65d0133f86b751 P 7bceff25285a4b1a818daf657bb0b467: No bootstrap required, opened a new log
I20260806 21:11:39.740039 20601 ts_tablet_manager.cc:1397] T 40b234b98a6445af8f65d0133f86b751 P 7bceff25285a4b1a818daf657bb0b467: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 21:11:39.740144 20603 ts_tablet_manager.cc:1397] T 40b234b98a6445af8f65d0133f86b751 P 947641bfc7f24bef9d652d2bded18339: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 21:11:39.740242 20605 tablet_bootstrap.cc:492] T 40b234b98a6445af8f65d0133f86b751 P b832bfe3f9af4b40bb1a84d3d2669c01: No bootstrap required, opened a new log
I20260806 21:11:39.740339 20605 ts_tablet_manager.cc:1397] T 40b234b98a6445af8f65d0133f86b751 P b832bfe3f9af4b40bb1a84d3d2669c01: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 21:11:39.740545 20601 raft_consensus.cc:348] T 40b234b98a6445af8f65d0133f86b751 P 7bceff25285a4b1a818daf657bb0b467 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } }
I20260806 21:11:39.740669 20601 raft_consensus.cc:374] T 40b234b98a6445af8f65d0133f86b751 P 7bceff25285a4b1a818daf657bb0b467 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:39.740624 20603 raft_consensus.cc:348] T 40b234b98a6445af8f65d0133f86b751 P 947641bfc7f24bef9d652d2bded18339 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } }
I20260806 21:11:39.740703 20601 raft_consensus.cc:729] T 40b234b98a6445af8f65d0133f86b751 P 7bceff25285a4b1a818daf657bb0b467 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bceff25285a4b1a818daf657bb0b467, State: Initialized, Role: FOLLOWER
I20260806 21:11:39.740739 20603 raft_consensus.cc:374] T 40b234b98a6445af8f65d0133f86b751 P 947641bfc7f24bef9d652d2bded18339 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:39.740769 20603 raft_consensus.cc:729] T 40b234b98a6445af8f65d0133f86b751 P 947641bfc7f24bef9d652d2bded18339 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 947641bfc7f24bef9d652d2bded18339, State: Initialized, Role: FOLLOWER
I20260806 21:11:39.740783 20605 raft_consensus.cc:348] T 40b234b98a6445af8f65d0133f86b751 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } }
I20260806 21:11:39.740886 20605 raft_consensus.cc:374] T 40b234b98a6445af8f65d0133f86b751 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:39.740844 20601 consensus_queue.cc:260] T 40b234b98a6445af8f65d0133f86b751 P 7bceff25285a4b1a818daf657bb0b467 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } }
I20260806 21:11:39.740916 20605 raft_consensus.cc:729] T 40b234b98a6445af8f65d0133f86b751 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b832bfe3f9af4b40bb1a84d3d2669c01, State: Initialized, Role: FOLLOWER
I20260806 21:11:39.741050 20605 consensus_queue.cc:260] T 40b234b98a6445af8f65d0133f86b751 P b832bfe3f9af4b40bb1a84d3d2669c01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } }
I20260806 21:11:39.741153 20601 ts_tablet_manager.cc:1428] T 40b234b98a6445af8f65d0133f86b751 P 7bceff25285a4b1a818daf657bb0b467: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 21:11:39.740937 20603 consensus_queue.cc:260] T 40b234b98a6445af8f65d0133f86b751 P 947641bfc7f24bef9d652d2bded18339 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } }
I20260806 21:11:39.741338 20605 ts_tablet_manager.cc:1428] T 40b234b98a6445af8f65d0133f86b751 P b832bfe3f9af4b40bb1a84d3d2669c01: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 21:11:39.741477 20603 ts_tablet_manager.cc:1428] T 40b234b98a6445af8f65d0133f86b751 P 947641bfc7f24bef9d652d2bded18339: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 21:11:39.756493 20611 raft_consensus.cc:482] T 40b234b98a6445af8f65d0133f86b751 P 947641bfc7f24bef9d652d2bded18339 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:11:39.756650 20611 raft_consensus.cc:504] T 40b234b98a6445af8f65d0133f86b751 P 947641bfc7f24bef9d652d2bded18339 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } }
I20260806 21:11:39.757815 20611 leader_election.cc:288] T 40b234b98a6445af8f65d0133f86b751 P 947641bfc7f24bef9d652d2bded18339 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b832bfe3f9af4b40bb1a84d3d2669c01 (127.16.178.1:35443), 7bceff25285a4b1a818daf657bb0b467 (127.16.178.2:39749)
I20260806 21:11:39.762182 20627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:39.758068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:35443 (local address 127.16.178.3:41629)
0806 21:11:39.758324 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:39.758342 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:39.758469 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:39.758908 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:39.758911 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:39.758933 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:11:39.759113 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:39.759121 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:39.760051 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:39.760055 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:39.761046 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:39.761056 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:39.761708 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:39.761711 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:39.761817 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:39.761842 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 21:11:39.761912 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 21:11:39.761989 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":23,"spinlock_wait_cycles":20224,"thread_start_us":81,"threads_started":1}
I20260806 21:11:39.762596 20628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:39.758156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:41629 (local address 127.16.178.1:35443)
0806 21:11:39.758428 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:39.758441 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:39.758456 (+ 15us) server_negotiation.cc:407] Connection header received
0806 21:11:39.758563 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:39.758566 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:39.758615 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:39.758690 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:11:39.759316 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:39.759932 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:39.761173 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:39.761594 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:39.761707 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:39.761767 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:39.762369 (+ 602us) server_negotiation.cc:299] Negotiation successful
0806 21:11:39.762456 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":98,"threads_started":1}
I20260806 21:11:39.763062 20631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:39.758482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.2:39749 (local address 127.16.178.3:58475)
0806 21:11:39.758833 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:39.758847 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:39.759016 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:39.759345 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:39.759348 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:39.759375 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:11:39.759488 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:39.759493 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:39.760555 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:39.760558 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:39.761795 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:39.761813 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:39.762640 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:39.762644 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:39.762748 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:39.762774 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 21:11:39.762846 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 21:11:39.762910 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"mutex_wait_us":31,"spinlock_wait_cycles":768,"thread_start_us":107,"threads_started":1}
I20260806 21:11:39.763080 20615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:39.758721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:58475 (local address 127.16.178.2:39749)
0806 21:11:39.758844 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:39.758849 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:39.758919 (+ 70us) server_negotiation.cc:407] Connection header received
0806 21:11:39.759130 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:39.759133 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:39.759177 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:39.759220 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:11:39.759601 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:39.760393 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:39.761941 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:39.762511 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:39.762630 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:39.762683 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:39.762878 (+ 195us) server_negotiation.cc:299] Negotiation successful
0806 21:11:39.762962 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 21:11:39.763197 19954 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40b234b98a6445af8f65d0133f86b751" candidate_uuid: "947641bfc7f24bef9d652d2bded18339" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" is_pre_election: true
I20260806 21:11:39.763370 19954 raft_consensus.cc:2454] T 40b234b98a6445af8f65d0133f86b751 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 947641bfc7f24bef9d652d2bded18339 in term 0.
I20260806 21:11:39.763593 20168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40b234b98a6445af8f65d0133f86b751" candidate_uuid: "947641bfc7f24bef9d652d2bded18339" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bceff25285a4b1a818daf657bb0b467" is_pre_election: true
I20260806 21:11:39.763736 20168 raft_consensus.cc:2454] T 40b234b98a6445af8f65d0133f86b751 P 7bceff25285a4b1a818daf657bb0b467 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 947641bfc7f24bef9d652d2bded18339 in term 0.
I20260806 21:11:39.763993 20274 leader_election.cc:302] T 40b234b98a6445af8f65d0133f86b751 P 947641bfc7f24bef9d652d2bded18339 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 947641bfc7f24bef9d652d2bded18339, b832bfe3f9af4b40bb1a84d3d2669c01; no voters:
I20260806 21:11:39.764292 20611 raft_consensus.cc:2781] T 40b234b98a6445af8f65d0133f86b751 P 947641bfc7f24bef9d652d2bded18339 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:11:39.764355 20611 raft_consensus.cc:482] T 40b234b98a6445af8f65d0133f86b751 P 947641bfc7f24bef9d652d2bded18339 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:11:39.764395 20611 raft_consensus.cc:3037] T 40b234b98a6445af8f65d0133f86b751 P 947641bfc7f24bef9d652d2bded18339 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:39.765434 20611 raft_consensus.cc:504] T 40b234b98a6445af8f65d0133f86b751 P 947641bfc7f24bef9d652d2bded18339 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } }
I20260806 21:11:39.765753 20611 leader_election.cc:288] T 40b234b98a6445af8f65d0133f86b751 P 947641bfc7f24bef9d652d2bded18339 [CANDIDATE]: Term 1 election: Requested vote from peers b832bfe3f9af4b40bb1a84d3d2669c01 (127.16.178.1:35443), 7bceff25285a4b1a818daf657bb0b467 (127.16.178.2:39749)
I20260806 21:11:39.766170 20168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40b234b98a6445af8f65d0133f86b751" candidate_uuid: "947641bfc7f24bef9d652d2bded18339" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bceff25285a4b1a818daf657bb0b467"
I20260806 21:11:39.766211 19954 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40b234b98a6445af8f65d0133f86b751" candidate_uuid: "947641bfc7f24bef9d652d2bded18339" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01"
I20260806 21:11:39.766307 20168 raft_consensus.cc:3037] T 40b234b98a6445af8f65d0133f86b751 P 7bceff25285a4b1a818daf657bb0b467 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:39.766335 19954 raft_consensus.cc:3037] T 40b234b98a6445af8f65d0133f86b751 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:39.767370 19954 raft_consensus.cc:2454] T 40b234b98a6445af8f65d0133f86b751 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 947641bfc7f24bef9d652d2bded18339 in term 1.
I20260806 21:11:39.767406 20168 raft_consensus.cc:2454] T 40b234b98a6445af8f65d0133f86b751 P 7bceff25285a4b1a818daf657bb0b467 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 947641bfc7f24bef9d652d2bded18339 in term 1.
I20260806 21:11:39.767679 20274 leader_election.cc:302] T 40b234b98a6445af8f65d0133f86b751 P 947641bfc7f24bef9d652d2bded18339 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bceff25285a4b1a818daf657bb0b467, 947641bfc7f24bef9d652d2bded18339; no voters:
I20260806 21:11:39.767920 20611 raft_consensus.cc:2781] T 40b234b98a6445af8f65d0133f86b751 P 947641bfc7f24bef9d652d2bded18339 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:11:39.768172 20611 raft_consensus.cc:686] T 40b234b98a6445af8f65d0133f86b751 P 947641bfc7f24bef9d652d2bded18339 [term 1 LEADER]: Becoming Leader. State: Replica: 947641bfc7f24bef9d652d2bded18339, State: Running, Role: LEADER
I20260806 21:11:39.768432 20611 consensus_queue.cc:237] T 40b234b98a6445af8f65d0133f86b751 P 947641bfc7f24bef9d652d2bded18339 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } }
I20260806 21:11:39.771054 19725 catalog_manager.cc:5495] T 40b234b98a6445af8f65d0133f86b751 P 947641bfc7f24bef9d652d2bded18339 reported cstate change: term changed from 0 to 1, leader changed from <none> to 947641bfc7f24bef9d652d2bded18339 (127.16.178.3). New cstate: current_term: 1 leader_uuid: "947641bfc7f24bef9d652d2bded18339" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "947641bfc7f24bef9d652d2bded18339" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42075 } health_report { overall_health: HEALTHY } } }
I20260806 21:11:39.787418 20637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:39.783005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39964 (local address 127.16.178.3:42075)
0806 21:11:39.783323 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:39.783332 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:39.783346 (+ 14us) server_negotiation.cc:407] Connection header received
0806 21:11:39.783446 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:39.783448 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:39.783515 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:39.783648 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:11:39.783837 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:39.784733 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:39.785541 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:39.785900 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:39.786001 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:39.786185 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:11:39.786790 (+ 605us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:11:39.786909 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:39.787003 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 21:11:39.787062 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":147,"threads_started":1}
I20260806 21:11:39.799543 20641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:39.795834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.2:39749 (local address 127.16.178.3:48981)
0806 21:11:39.796279 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:39.796292 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:39.796400 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:39.796625 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:39.796628 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:39.796636 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 21:11:39.796757 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:39.796764 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:39.797723 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:39.797725 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:39.798339 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:39.798346 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:39.798799 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:39.798803 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:39.798891 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:39.798947 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 21:11:39.799229 (+ 282us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 21:11:39.799236 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 21:11:39.799294 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 21:11:39.799373 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":114,"threads_started":1}
I20260806 21:11:39.799530 20615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:39.796026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:48981 (local address 127.16.178.2:39749)
0806 21:11:39.796179 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:39.796184 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:39.796362 (+ 178us) server_negotiation.cc:407] Connection header received
0806 21:11:39.796490 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:39.796493 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:39.796547 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:39.796608 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:11:39.796894 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:39.797601 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:39.798458 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:39.798675 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:39.798766 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:39.799079 (+ 313us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:11:39.799158 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:11:39.799216 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:39.799310 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 21:11:39.799366 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 21:11:39.806025 19954 raft_consensus.cc:1261] T 40b234b98a6445af8f65d0133f86b751 P b832bfe3f9af4b40bb1a84d3d2669c01 [term 1 FOLLOWER]: Refusing update from remote peer 947641bfc7f24bef9d652d2bded18339: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:11:39.806107 20168 raft_consensus.cc:1261] T 40b234b98a6445af8f65d0133f86b751 P 7bceff25285a4b1a818daf657bb0b467 [term 1 FOLLOWER]: Refusing update from remote peer 947641bfc7f24bef9d652d2bded18339: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:11:39.806497 20632 consensus_queue.cc:1035] T 40b234b98a6445af8f65d0133f86b751 P 947641bfc7f24bef9d652d2bded18339 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b832bfe3f9af4b40bb1a84d3d2669c01" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:11:39.806666 20611 consensus_queue.cc:1035] T 40b234b98a6445af8f65d0133f86b751 P 947641bfc7f24bef9d652d2bded18339 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bceff25285a4b1a818daf657bb0b467" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:11:39.825914 20637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:39.822380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:34707 (local address 127.16.178.3:42075)
0806 21:11:39.822701 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:39.822705 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:39.822717 (+ 12us) server_negotiation.cc:407] Connection header received
0806 21:11:39.822760 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:39.822763 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:39.822809 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:39.822914 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:11:39.823152 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:39.823899 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:39.824851 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:39.825140 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:39.825243 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:39.825490 (+ 247us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:11:39.825577 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:11:39.825647 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:39.825753 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 21:11:39.825803 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244}
I20260806 21:11:39.825956 20619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:39.822311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.3:42075 (local address 127.16.178.2:34707)
0806 21:11:39.822434 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:39.822450 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:39.822523 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:39.822939 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:39.822943 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:39.822951 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 21:11:39.823059 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:39.823064 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:39.823989 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:39.823992 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:39.824736 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:39.824747 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:39.825235 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:39.825238 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:39.825321 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:39.825354 (+ 33us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 21:11:39.825668 (+ 314us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 21:11:39.825675 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 21:11:39.825742 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 21:11:39.825823 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
W20260806 21:11:39.826685 20556 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 21:11:41.392640 19713 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:11:41.392812 19713 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:11:41.395463 19713 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a6079d59bd5d4b769d0ff72bdbd21121: Sending DeleteTablet for 3 replicas of tablet 40b234b98a6445af8f65d0133f86b751
I20260806 21:11:41.396132 19928 tablet_service.cc:1514] Processing DeleteTablet for tablet 40b234b98a6445af8f65d0133f86b751 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:11:41 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:42664
I20260806 21:11:41.396139 20134 tablet_service.cc:1514] Processing DeleteTablet for tablet 40b234b98a6445af8f65d0133f86b751 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:11:41 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:40140
I20260806 21:11:41.396263 20342 tablet_service.cc:1514] Processing DeleteTablet for tablet 40b234b98a6445af8f65d0133f86b751 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:11:41 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:53238
I20260806 21:11:41.396457 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 19834
I20260806 21:11:41.412351 20674 tablet_replica.cc:330] T 40b234b98a6445af8f65d0133f86b751 P 7bceff25285a4b1a818daf657bb0b467: stopping tablet replica
I20260806 21:11:41.412632 20674 raft_consensus.cc:2229] T 40b234b98a6445af8f65d0133f86b751 P 7bceff25285a4b1a818daf657bb0b467 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:11:41.412756 20675 tablet_replica.cc:330] T 40b234b98a6445af8f65d0133f86b751 P 947641bfc7f24bef9d652d2bded18339: stopping tablet replica
I20260806 21:11:41.412843 20674 raft_consensus.cc:2258] T 40b234b98a6445af8f65d0133f86b751 P 7bceff25285a4b1a818daf657bb0b467 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:11:41.412933 20675 raft_consensus.cc:2229] T 40b234b98a6445af8f65d0133f86b751 P 947641bfc7f24bef9d652d2bded18339 [term 1 LEADER]: Raft consensus shutting down.
I20260806 21:11:41.413360 20675 raft_consensus.cc:2258] T 40b234b98a6445af8f65d0133f86b751 P 947641bfc7f24bef9d652d2bded18339 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:11:41.413859 20674 ts_tablet_manager.cc:1905] T 40b234b98a6445af8f65d0133f86b751 P 7bceff25285a4b1a818daf657bb0b467: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:11:41.413928 20675 ts_tablet_manager.cc:1905] T 40b234b98a6445af8f65d0133f86b751 P 947641bfc7f24bef9d652d2bded18339: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:11:41.416661 20674 ts_tablet_manager.cc:1918] T 40b234b98a6445af8f65d0133f86b751 P 7bceff25285a4b1a818daf657bb0b467: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 21:11:41.416661 20675 ts_tablet_manager.cc:1918] T 40b234b98a6445af8f65d0133f86b751 P 947641bfc7f24bef9d652d2bded18339: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 21:11:41.416747 20674 log.cc:1196] T 40b234b98a6445af8f65d0133f86b751 P 7bceff25285a4b1a818daf657bb0b467: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-1/wal/wals/40b234b98a6445af8f65d0133f86b751
I20260806 21:11:41.416747 20675 log.cc:1196] T 40b234b98a6445af8f65d0133f86b751 P 947641bfc7f24bef9d652d2bded18339: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786050686797304-17096-0/minicluster-data/ts-2/wal/wals/40b234b98a6445af8f65d0133f86b751
I20260806 21:11:41.417037 20674 ts_tablet_manager.cc:1939] T 40b234b98a6445af8f65d0133f86b751 P 7bceff25285a4b1a818daf657bb0b467: Deleting consensus metadata
I20260806 21:11:41.417038 20675 ts_tablet_manager.cc:1939] T 40b234b98a6445af8f65d0133f86b751 P 947641bfc7f24bef9d652d2bded18339: Deleting consensus metadata
I20260806 21:11:41.417861 19704 catalog_manager.cc:4841] TS 7bceff25285a4b1a818daf657bb0b467 (127.16.178.2:39749): tablet 40b234b98a6445af8f65d0133f86b751 (table test-table [id=72f901e2e1934bd391407b4afcf4c38a]) successfully deleted
I20260806 21:11:41.418040 19704 catalog_manager.cc:4841] TS 947641bfc7f24bef9d652d2bded18339 (127.16.178.3:42075): tablet 40b234b98a6445af8f65d0133f86b751 (table test-table [id=72f901e2e1934bd391407b4afcf4c38a]) successfully deleted
W20260806 21:11:41.418506 19704 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.178.1:35443 (error 108)
I20260806 21:11:41.419365 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 20039
I20260806 21:11:41.419730 20678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:41.419279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:35443 (local address 127.0.0.1:49604)
0806 21:11:41.419542 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:41.419624 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.1:35443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":76,"threads_started":1}
W20260806 21:11:41.424005 19704 catalog_manager.cc:4584] TS b832bfe3f9af4b40bb1a84d3d2669c01 (127.16.178.1:35443): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 40b234b98a6445af8f65d0133f86b751: Network error: Client connection negotiation failed: client connection to 127.16.178.1:35443: connect: Connection refused (error 111)
I20260806 21:11:41.432603 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 20247
I20260806 21:11:41.448635 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 19678
2026-08-06T21:11:41Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6379 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:11:41 dist-test-slave-lgr5 krb5kdc[20681](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:11:41 dist-test-slave-lgr5 krb5kdc[20681](info): set up 2 sockets
Aug 06 21:11:41 dist-test-slave-lgr5 krb5kdc[20681](info): commencing operation
krb5kdc: starting...
W20260806 21:11:42.609668 17096 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Aug 06 21:11:42 dist-test-slave-lgr5 krb5kdc[20681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:11:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:11:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.178.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
I20260806 21:11:42.742658 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:32843
--webserver_interface=127.16.178.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:34647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:32843
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:11:42.863291 20699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:42.863615 20699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:42.863672 20699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:42.867766 20699 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:11:42.867851 20699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:42.867877 20699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:42.867898 20699 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:11:42.867920 20699 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:11:42.873459 20699 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:34647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:32843
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:32843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.178.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20699
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:42.874572 20699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:42.875628 20699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 21:11:42.884721 20699 server_base.cc:1034] running on GCE node
W20260806 21:11:42.884665 20710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:42.884662 20708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:42.884665 20706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:42.885533 20699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:42.886677 20699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:42.887844 20699 hybrid_clock.cc:648] HybridClock initialized: now 1786050702887830 us; error 27 us; skew 500 ppm
Aug 06 21:11:42 dist-test-slave-lgr5 krb5kdc[20681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050702, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:42.896028 20699 init.cc:377] Logged in from keytab as kudu/127.16.178.62@KRBTEST.COM (short username kudu)
I20260806 21:11:42.897063 20699 webserver.cc:460] Webserver started at http://127.16.178.62:40175/ using document root <none> and password file <none>
I20260806 21:11:42.897629 20699 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:42.897694 20699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:42.897910 20699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:42.900141 20699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "f44bfd939c0246b6bb914152e206135e"
format_stamp: "Formatted at 2026-08-06 21:11:42 on dist-test-slave-lgr5"
server_key: "c6dcdcaa4709f9202ce2ef6dbf5d2b37"
server_key_iv: "121a3b7d0b528885f950c66828ed884b"
server_key_version: "encryptionkey@0"
I20260806 21:11:42.900739 20699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "f44bfd939c0246b6bb914152e206135e"
format_stamp: "Formatted at 2026-08-06 21:11:42 on dist-test-slave-lgr5"
server_key: "c6dcdcaa4709f9202ce2ef6dbf5d2b37"
server_key_iv: "121a3b7d0b528885f950c66828ed884b"
server_key_version: "encryptionkey@0"
I20260806 21:11:42.904570 20699 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:11:42.907147 20722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:42.908541 20699 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:11:42.908684 20699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "f44bfd939c0246b6bb914152e206135e"
format_stamp: "Formatted at 2026-08-06 21:11:42 on dist-test-slave-lgr5"
server_key: "c6dcdcaa4709f9202ce2ef6dbf5d2b37"
server_key_iv: "121a3b7d0b528885f950c66828ed884b"
server_key_version: "encryptionkey@0"
I20260806 21:11:42.908803 20699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:42.927340 20699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:42.931349 20699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:42.931603 20699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:42.938839 20699 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.62:32843
I20260806 21:11:42.938856 20799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.62:32843 every 8 connection(s)
I20260806 21:11:42.939845 20699 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:11:42.940659 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 20699
I20260806 21:11:42.940775 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:11:42.941067 17096 external_mini_cluster.cc:1428] Setting key ecf6f6806d23d30a06c8c5479577011d
I20260806 21:11:42.943342 20800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:42.950584 20800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f44bfd939c0246b6bb914152e206135e: Bootstrap starting.
Aug 06 21:11:42 dist-test-slave-lgr5 krb5kdc[20681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:11:42.953012 20800 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f44bfd939c0246b6bb914152e206135e: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:42.953945 20800 log.cc:824] T 00000000000000000000000000000000 P f44bfd939c0246b6bb914152e206135e: Log is configured to *not* fsync() on all Append() calls
I20260806 21:11:42.956410 20800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f44bfd939c0246b6bb914152e206135e: No bootstrap required, opened a new log
I20260806 21:11:42.956465 20804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:42.942617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34332 (local address 127.16.178.62:32843)
0806 21:11:42.943036 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:42.943051 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:42.943090 (+ 39us) server_negotiation.cc:407] Connection header received
0806 21:11:42.943775 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:42.943795 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:42.944183 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:42.944490 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:42.945450 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:42.946682 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:42.947886 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:42.948447 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:42.948596 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:42.951442 (+ 2846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:42.951448 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:42.951462 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:42.951492 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:42.953144 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:42.953685 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:42.953691 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:42.953696 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:42.953814 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:42.954157 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:42.954162 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:42.954164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:42.954610 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:42.954784 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:42.955140 (+ 356us) server_negotiation.cc:299] Negotiation successful
0806 21:11:42.955458 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":116,"threads_started":1}
I20260806 21:11:42.960487 20800 raft_consensus.cc:348] T 00000000000000000000000000000000 P f44bfd939c0246b6bb914152e206135e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f44bfd939c0246b6bb914152e206135e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 32843 } }
I20260806 21:11:42.960745 20800 raft_consensus.cc:374] T 00000000000000000000000000000000 P f44bfd939c0246b6bb914152e206135e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:42.960780 20800 raft_consensus.cc:729] T 00000000000000000000000000000000 P f44bfd939c0246b6bb914152e206135e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f44bfd939c0246b6bb914152e206135e, State: Initialized, Role: FOLLOWER
I20260806 21:11:42.961179 20800 consensus_queue.cc:260] T 00000000000000000000000000000000 P f44bfd939c0246b6bb914152e206135e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f44bfd939c0246b6bb914152e206135e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 32843 } }
I20260806 21:11:42.961308 20800 raft_consensus.cc:388] T 00000000000000000000000000000000 P f44bfd939c0246b6bb914152e206135e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:11:42.961365 20800 raft_consensus.cc:482] T 00000000000000000000000000000000 P f44bfd939c0246b6bb914152e206135e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:11:42.961449 20800 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f44bfd939c0246b6bb914152e206135e [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:42.962395 20800 raft_consensus.cc:504] T 00000000000000000000000000000000 P f44bfd939c0246b6bb914152e206135e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f44bfd939c0246b6bb914152e206135e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 32843 } }
I20260806 21:11:42.962731 20800 leader_election.cc:302] T 00000000000000000000000000000000 P f44bfd939c0246b6bb914152e206135e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f44bfd939c0246b6bb914152e206135e; no voters:
I20260806 21:11:42.963048 20800 leader_election.cc:288] T 00000000000000000000000000000000 P f44bfd939c0246b6bb914152e206135e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:11:42.963217 20808 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f44bfd939c0246b6bb914152e206135e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:11:42.963546 20808 raft_consensus.cc:686] T 00000000000000000000000000000000 P f44bfd939c0246b6bb914152e206135e [term 1 LEADER]: Becoming Leader. State: Replica: f44bfd939c0246b6bb914152e206135e, State: Running, Role: LEADER
I20260806 21:11:42.963904 20808 consensus_queue.cc:237] T 00000000000000000000000000000000 P f44bfd939c0246b6bb914152e206135e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f44bfd939c0246b6bb914152e206135e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 32843 } }
I20260806 21:11:42.964485 20800 sys_catalog.cc:564] T 00000000000000000000000000000000 P f44bfd939c0246b6bb914152e206135e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:11:42.965539 20809 sys_catalog.cc:455] T 00000000000000000000000000000000 P f44bfd939c0246b6bb914152e206135e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f44bfd939c0246b6bb914152e206135e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f44bfd939c0246b6bb914152e206135e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 32843 } } }
I20260806 21:11:42.965687 20809 sys_catalog.cc:458] T 00000000000000000000000000000000 P f44bfd939c0246b6bb914152e206135e [sys.catalog]: This master's current role is: LEADER
I20260806 21:11:42.966076 20810 sys_catalog.cc:455] T 00000000000000000000000000000000 P f44bfd939c0246b6bb914152e206135e [sys.catalog]: SysCatalogTable state changed. Reason: New leader f44bfd939c0246b6bb914152e206135e. Latest consensus state: current_term: 1 leader_uuid: "f44bfd939c0246b6bb914152e206135e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f44bfd939c0246b6bb914152e206135e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 32843 } } }
I20260806 21:11:42.966243 20810 sys_catalog.cc:458] T 00000000000000000000000000000000 P f44bfd939c0246b6bb914152e206135e [sys.catalog]: This master's current role is: LEADER
W20260806 21:11:42.969524 20828 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f44bfd939c0246b6bb914152e206135e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 21:11:42.969615 20828 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 21:11:42.969703 20819 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:11:42.970458 20819 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:11:42.977138 20819 catalog_manager.cc:1347] Generated new cluster ID: 8f3985f39e66468589c957fea63c5a0c
I20260806 21:11:42.977246 20819 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:11:42.993343 20819 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:11:42.995568 20819 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:11:43.010123 20819 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f44bfd939c0246b6bb914152e206135e: Generated new TSK 0
I20260806 21:11:43.011237 20819 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.178.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab.
Entry for principal kudu/127.16.178.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab.
I20260806 21:11:43.051997 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.1:0
--local_ip_for_outbound_sockets=127.16.178.1
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:32843
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab
--principal=kudu/127.16.178.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:34647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:11:43.172472 20837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:43.172796 20837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:43.172842 20837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:43.177112 20837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:43.177194 20837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:43.177256 20837 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.1
I20260806 21:11:43.182945 20837 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:34647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab
--principal=kudu/127.16.178.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:32843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.1
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:43.184398 20837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:43.185581 20837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:43.196234 20849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:43.196237 20846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:43.196278 20837 server_base.cc:1034] running on GCE node
W20260806 21:11:43.196234 20853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:43.197036 20837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:43.197733 20837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:43.198907 20837 hybrid_clock.cc:648] HybridClock initialized: now 1786050703198883 us; error 36 us; skew 500 ppm
Aug 06 21:11:43 dist-test-slave-lgr5 krb5kdc[20681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050703, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:43.201829 20837 init.cc:377] Logged in from keytab as kudu/127.16.178.1@KRBTEST.COM (short username kudu)
I20260806 21:11:43.202749 20837 webserver.cc:460] Webserver started at http://127.16.178.1:44553/ using document root <none> and password file <none>
I20260806 21:11:43.203306 20837 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:43.203366 20837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:43.203545 20837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:43.205726 20837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-0/data/instance:
uuid: "ea0ce15f795545d69200c53f722eb381"
format_stamp: "Formatted at 2026-08-06 21:11:43 on dist-test-slave-lgr5"
server_key: "86e1797129f34c774f3a1f996ac0ac89"
server_key_iv: "582478870cfbaebd2ed3491dee79d3b7"
server_key_version: "encryptionkey@0"
I20260806 21:11:43.206288 20837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance:
uuid: "ea0ce15f795545d69200c53f722eb381"
format_stamp: "Formatted at 2026-08-06 21:11:43 on dist-test-slave-lgr5"
server_key: "86e1797129f34c774f3a1f996ac0ac89"
server_key_iv: "582478870cfbaebd2ed3491dee79d3b7"
server_key_version: "encryptionkey@0"
I20260806 21:11:43.210244 20837 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:11:43.213017 20863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:43.214391 20837 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:11:43.214514 20837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-0/wal
uuid: "ea0ce15f795545d69200c53f722eb381"
format_stamp: "Formatted at 2026-08-06 21:11:43 on dist-test-slave-lgr5"
server_key: "86e1797129f34c774f3a1f996ac0ac89"
server_key_iv: "582478870cfbaebd2ed3491dee79d3b7"
server_key_version: "encryptionkey@0"
I20260806 21:11:43.214619 20837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:43.228866 20837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:43.232156 20837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:43.232347 20837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:43.232980 20837 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:11:43.233911 20837 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:11:43.233978 20837 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:43.234025 20837 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:11:43.234050 20837 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:43.243768 20837 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.1:45485
I20260806 21:11:43.243821 21028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.1:45485 every 8 connection(s)
I20260806 21:11:43.244817 20837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
I20260806 21:11:43.250211 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 20837
I20260806 21:11:43.250355 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance
I20260806 21:11:43.250653 17096 external_mini_cluster.cc:1428] Setting key accb535b03d9665d651035b340ea86a3
Aug 06 21:11:43 dist-test-slave-lgr5 krb5kdc[20681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050703, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.178.2@KRBTEST.COM; defaulting to no policy
I20260806 21:11:43.260883 20804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:43.246946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:48775 (local address 127.16.178.62:32843)
0806 21:11:43.247139 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:43.247143 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:43.247950 (+ 807us) server_negotiation.cc:407] Connection header received
0806 21:11:43.249252 (+ 1302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:43.249255 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:43.249317 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:43.249402 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:43.250863 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:43.251501 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:43.252868 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:43.253109 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:43.253202 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:43.256552 (+ 3350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:43.256556 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:43.256558 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:43.256593 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:43.258265 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:43.258965 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:43.258969 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:43.258970 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:43.259067 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:43.259485 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:43.259488 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:43.259489 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:43.259723 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:43.259835 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:43.260496 (+ 661us) server_negotiation.cc:299] Negotiation successful
0806 21:11:43.260670 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 21:11:43.261757 21034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:43.247301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:32843 (local address 127.16.178.1:48775)
0806 21:11:43.247780 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:43.247831 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:43.249035 (+ 1204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:43.249606 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:43.249619 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:43.250046 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:43.250671 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:43.250695 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:43.251634 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:43.251637 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:43.252722 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:43.252736 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:43.253273 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:43.253278 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:43.253470 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:43.254173 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:43.254192 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:43.256234 (+ 2042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:43.258435 (+ 2201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:43.258444 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:43.258468 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:43.258796 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:43.259169 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:43.259172 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:43.259175 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:43.259366 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:43.259837 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:43.259846 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:43.260205 (+ 359us) client_negotiation.cc:769] Sending connection context
0806 21:11:43.260411 (+ 206us) client_negotiation.cc:240] Negotiation successful
0806 21:11:43.260811 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":107,"threads_started":1}
I20260806 21:11:43.263329 21029 heartbeater.cc:347] Connected to a master server at 127.16.178.62:32843
I20260806 21:11:43.263612 21029 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.2@KRBTEST.COM" created.
I20260806 21:11:43.264362 21029 heartbeater.cc:511] Master 127.16.178.62:32843 requested a full tablet report, sending...
I20260806 21:11:43.266278 20750 ts_manager.cc:194] Registered new tserver with Master: ea0ce15f795545d69200c53f722eb381 (127.16.178.1:45485)
I20260806 21:11:43.267457 20750 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.1@KRBTEST.COM'} at 127.16.178.1:48775
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab.
Entry for principal kudu/127.16.178.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab.
I20260806 21:11:43.281509 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.2:0
--local_ip_for_outbound_sockets=127.16.178.2
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:32843
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab
--principal=kudu/127.16.178.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:34647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:11:43.399017 21038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:43.399284 21038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:43.399327 21038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:43.403337 21038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:43.403398 21038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:43.403455 21038 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.2
I20260806 21:11:43.408947 21038 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:34647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab
--principal=kudu/127.16.178.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:32843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21038
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.2
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:43.409981 21038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:43.410964 21038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:43.420933 21045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:43.420933 21051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:43.420933 21044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:43.421828 21038 server_base.cc:1034] running on GCE node
I20260806 21:11:43.422312 21038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:43.423051 21038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:43.424213 21038 hybrid_clock.cc:648] HybridClock initialized: now 1786050703424182 us; error 38 us; skew 500 ppm
Aug 06 21:11:43 dist-test-slave-lgr5 krb5kdc[20681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050703, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:43.427006 21038 init.cc:377] Logged in from keytab as kudu/127.16.178.2@KRBTEST.COM (short username kudu)
I20260806 21:11:43.428012 21038 webserver.cc:460] Webserver started at http://127.16.178.2:38643/ using document root <none> and password file <none>
I20260806 21:11:43.428637 21038 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:43.428709 21038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:43.428925 21038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:43.431133 21038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-1/data/instance:
uuid: "0115da56b4d64bb2a5cf9e6156ed8b5d"
format_stamp: "Formatted at 2026-08-06 21:11:43 on dist-test-slave-lgr5"
server_key: "3980619c77045f27b94f6a95db1c49ef"
server_key_iv: "fbcffa781e05576a0cf605b7b4583cfa"
server_key_version: "encryptionkey@0"
I20260806 21:11:43.431733 21038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance:
uuid: "0115da56b4d64bb2a5cf9e6156ed8b5d"
format_stamp: "Formatted at 2026-08-06 21:11:43 on dist-test-slave-lgr5"
server_key: "3980619c77045f27b94f6a95db1c49ef"
server_key_iv: "fbcffa781e05576a0cf605b7b4583cfa"
server_key_version: "encryptionkey@0"
I20260806 21:11:43.436062 21038 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260806 21:11:43.438868 21064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:43.440356 21038 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 21:11:43.440495 21038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-1/wal
uuid: "0115da56b4d64bb2a5cf9e6156ed8b5d"
format_stamp: "Formatted at 2026-08-06 21:11:43 on dist-test-slave-lgr5"
server_key: "3980619c77045f27b94f6a95db1c49ef"
server_key_iv: "fbcffa781e05576a0cf605b7b4583cfa"
server_key_version: "encryptionkey@0"
I20260806 21:11:43.440623 21038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:43.454124 21038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:43.457587 21038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:43.457778 21038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:43.458386 21038 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:11:43.459394 21038 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:11:43.459445 21038 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:43.459494 21038 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:11:43.459523 21038 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:43.469020 21038 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.2:34819
I20260806 21:11:43.469053 21235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.2:34819 every 8 connection(s)
I20260806 21:11:43.469996 21038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
I20260806 21:11:43.479245 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 21038
I20260806 21:11:43.479377 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance
I20260806 21:11:43.479648 17096 external_mini_cluster.cc:1428] Setting key 13aa4bb65d2e750d936540bff13663c5
Aug 06 21:11:43 dist-test-slave-lgr5 krb5kdc[20681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050703, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.2@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:11:43.484400 20804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:43.471940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:46107 (local address 127.16.178.62:32843)
0806 21:11:43.472125 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:43.472129 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:43.472967 (+ 838us) server_negotiation.cc:407] Connection header received
0806 21:11:43.474290 (+ 1323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:43.474294 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:43.474354 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:43.474426 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:43.475703 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:43.476278 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:43.477458 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:43.477687 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:43.477778 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:43.480551 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:43.480556 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:43.480558 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:43.480584 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:43.482062 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:43.482724 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:43.482727 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:43.482728 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:43.482814 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:43.483188 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:43.483191 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:43.483192 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:43.483375 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:43.483488 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:43.484069 (+ 581us) server_negotiation.cc:299] Negotiation successful
0806 21:11:43.484231 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 21:11:43.485463 21238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:43.472330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:32843 (local address 127.16.178.2:46107)
0806 21:11:43.472813 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:43.472860 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:43.474052 (+ 1192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:43.474632 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:43.474639 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:43.475003 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:43.475553 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:43.475568 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:43.476409 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:43.476412 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:43.477317 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:43.477327 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:43.477845 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:43.477850 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:43.478018 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:43.478580 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:43.478598 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:43.480343 (+ 1745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:43.482189 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:43.482196 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:43.482217 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:43.482602 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:43.482899 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:43.482902 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:43.482904 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:43.483096 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:43.483493 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:43.483501 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:43.483822 (+ 321us) client_negotiation.cc:769] Sending connection context
0806 21:11:43.483997 (+ 175us) client_negotiation.cc:240] Negotiation successful
0806 21:11:43.484351 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":111,"threads_started":1}
I20260806 21:11:43.486739 21236 heartbeater.cc:347] Connected to a master server at 127.16.178.62:32843
I20260806 21:11:43.487007 21236 heartbeater.cc:464] Registering TS with master...
I20260806 21:11:43.487572 21236 heartbeater.cc:511] Master 127.16.178.62:32843 requested a full tablet report, sending...
I20260806 21:11:43.488739 20750 ts_manager.cc:194] Registered new tserver with Master: 0115da56b4d64bb2a5cf9e6156ed8b5d (127.16.178.2:34819)
I20260806 21:11:43.489326 20750 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.2@KRBTEST.COM'} at 127.16.178.2:46107
WARNING: no policy specified for kudu/127.16.178.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab.
Entry for principal kudu/127.16.178.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab.
I20260806 21:11:43.511082 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.3:0
--local_ip_for_outbound_sockets=127.16.178.3
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:32843
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab
--principal=kudu/127.16.178.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:34647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:11:43.628376 21245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:43.628679 21245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:43.628743 21245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:43.632933 21245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:43.633020 21245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:43.633104 21245 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.3
I20260806 21:11:43.638692 21245 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:34647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab
--principal=kudu/127.16.178.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:32843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21245
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.3
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:43.639856 21245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:43.640988 21245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:43.651225 21255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:43.651225 21252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:43.651312 21245 server_base.cc:1034] running on GCE node
W20260806 21:11:43.651263 21253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:43.651913 21245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:43.652709 21245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:43.653900 21245 hybrid_clock.cc:648] HybridClock initialized: now 1786050703653871 us; error 46 us; skew 500 ppm
Aug 06 21:11:43 dist-test-slave-lgr5 krb5kdc[20681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050703, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:43.656877 21245 init.cc:377] Logged in from keytab as kudu/127.16.178.3@KRBTEST.COM (short username kudu)
I20260806 21:11:43.657905 21245 webserver.cc:460] Webserver started at http://127.16.178.3:37843/ using document root <none> and password file <none>
I20260806 21:11:43.658480 21245 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:43.658552 21245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:43.658767 21245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:43.661159 21245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-2/data/instance:
uuid: "754e34a1a9bd456cbbcb8314a699d328"
format_stamp: "Formatted at 2026-08-06 21:11:43 on dist-test-slave-lgr5"
server_key: "3de0781283db4b4678434324645f6d9d"
server_key_iv: "18ae31684cb148c840a4ec3f11ba7071"
server_key_version: "encryptionkey@0"
I20260806 21:11:43.661780 21245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance:
uuid: "754e34a1a9bd456cbbcb8314a699d328"
format_stamp: "Formatted at 2026-08-06 21:11:43 on dist-test-slave-lgr5"
server_key: "3de0781283db4b4678434324645f6d9d"
server_key_iv: "18ae31684cb148c840a4ec3f11ba7071"
server_key_version: "encryptionkey@0"
I20260806 21:11:43.666327 21245 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:11:43.669147 21265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:43.670477 21245 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:11:43.670614 21245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-2/wal
uuid: "754e34a1a9bd456cbbcb8314a699d328"
format_stamp: "Formatted at 2026-08-06 21:11:43 on dist-test-slave-lgr5"
server_key: "3de0781283db4b4678434324645f6d9d"
server_key_iv: "18ae31684cb148c840a4ec3f11ba7071"
server_key_version: "encryptionkey@0"
I20260806 21:11:43.670727 21245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:43.686070 21245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:43.689183 21245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:43.689373 21245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:43.689986 21245 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:11:43.690873 21245 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:11:43.690925 21245 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:43.690984 21245 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:11:43.691018 21245 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:43.699862 21245 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.3:35193
I20260806 21:11:43.699885 21439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.3:35193 every 8 connection(s)
I20260806 21:11:43.700961 21245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
I20260806 21:11:43.709290 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 21245
I20260806 21:11:43.709440 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance
I20260806 21:11:43.709749 17096 external_mini_cluster.cc:1428] Setting key 17ca5238a9f1616c5269690e4e7547b7
Aug 06 21:11:43 dist-test-slave-lgr5 krb5kdc[20681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050703, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.3@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:11:43.715634 20804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:43.702957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:39051 (local address 127.16.178.62:32843)
0806 21:11:43.703151 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:43.703156 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:43.704123 (+ 967us) server_negotiation.cc:407] Connection header received
0806 21:11:43.705435 (+ 1312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:43.705438 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:43.705495 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:43.705572 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:43.706803 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:43.707343 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:43.708632 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:43.708858 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:43.708940 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:43.711835 (+ 2895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:43.711839 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:43.711842 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:43.711874 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:43.713406 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:43.713974 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:43.713978 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:43.713980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:43.714077 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:43.714426 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:43.714430 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:43.714431 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:43.714640 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:43.714730 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:43.715321 (+ 591us) server_negotiation.cc:299] Negotiation successful
0806 21:11:43.715460 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 21:11:43.716612 21445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:43.703412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:32843 (local address 127.16.178.3:39051)
0806 21:11:43.703923 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:43.703972 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:43.705220 (+ 1248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:43.705746 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:43.705753 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:43.706132 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:43.706680 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:43.706694 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:43.707502 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:43.707505 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:43.708511 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:43.708524 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:43.709004 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:43.709009 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:43.709218 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:43.709912 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:43.709934 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:43.711652 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:43.713548 (+ 1896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:43.713556 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:43.713577 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:43.713873 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:43.714168 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:43.714171 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:43.714173 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:43.714335 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:43.714748 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:43.714756 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:43.715068 (+ 312us) client_negotiation.cc:769] Sending connection context
0806 21:11:43.715260 (+ 192us) client_negotiation.cc:240] Negotiation successful
0806 21:11:43.715657 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":118,"threads_started":1}
I20260806 21:11:43.718021 21442 heartbeater.cc:347] Connected to a master server at 127.16.178.62:32843
I20260806 21:11:43.718279 21442 heartbeater.cc:464] Registering TS with master...
I20260806 21:11:43.718892 21442 heartbeater.cc:511] Master 127.16.178.62:32843 requested a full tablet report, sending...
I20260806 21:11:43.720161 20750 ts_manager.cc:194] Registered new tserver with Master: 754e34a1a9bd456cbbcb8314a699d328 (127.16.178.3:35193)
I20260806 21:11:43.720777 20750 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.3@KRBTEST.COM'} at 127.16.178.3:39051
I20260806 21:11:43.724046 17096 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 21:11:43.739432 20804 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:11:43.737867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34348 (local address 127.16.178.62:32843)
0806 21:11:43.738042 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:43.738047 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:43.738160 (+ 113us) server_negotiation.cc:407] Connection header received
0806 21:11:43.738289 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:43.738291 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:43.738337 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:43.738408 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:43.739271 (+ 863us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34348: BlockingRecv error: recv got EOF from 127.0.0.1:34348 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 21:11:43.741468 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 20837
I20260806 21:11:43.769057 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 21038
I20260806 21:11:43.782155 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 21245
I20260806 21:11:43.797178 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 20699
2026-08-06T21:11:43Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2350 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050686797304-17096-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:11:43 dist-test-slave-lgr5 krb5kdc[21458](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:11:43 dist-test-slave-lgr5 krb5kdc[21458](info): set up 2 sockets
Aug 06 21:11:43 dist-test-slave-lgr5 krb5kdc[21458](info): commencing operation
krb5kdc: starting...
W20260806 21:11:44.951784 17096 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Aug 06 21:11:45 dist-test-slave-lgr5 krb5kdc[21458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:11:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:11:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.178.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
I20260806 21:11:45.082979 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:39143
--webserver_interface=127.16.178.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:45831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:39143
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:11:45.199247 21477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:45.199566 21477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:45.199626 21477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:45.203750 21477 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:11:45.203828 21477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:45.203856 21477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:45.203878 21477 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:11:45.203897 21477 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:11:45.209376 21477 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:45831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:39143
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:39143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.178.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:45.210490 21477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:45.211546 21477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:45.219988 21488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:45.219980 21492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:45.219956 21487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:45.220165 21477 server_base.cc:1034] running on GCE node
I20260806 21:11:45.220888 21477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:45.222007 21477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:45.223188 21477 hybrid_clock.cc:648] HybridClock initialized: now 1786050705223167 us; error 34 us; skew 500 ppm
Aug 06 21:11:45 dist-test-slave-lgr5 krb5kdc[21458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050705, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:45.226032 21477 init.cc:377] Logged in from keytab as kudu/127.16.178.62@KRBTEST.COM (short username kudu)
I20260806 21:11:45.227039 21477 webserver.cc:460] Webserver started at http://127.16.178.62:46219/ using document root <none> and password file <none>
I20260806 21:11:45.227625 21477 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:45.227698 21477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:45.227907 21477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:45.230084 21477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "3687dcadd3964dd3ac2323f92e2edee8"
format_stamp: "Formatted at 2026-08-06 21:11:45 on dist-test-slave-lgr5"
server_key: "b94d6915a68405df06954472fb1b1889"
server_key_iv: "079b5d9661790d79a5634d9b7c308b5f"
server_key_version: "encryptionkey@0"
I20260806 21:11:45.230660 21477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "3687dcadd3964dd3ac2323f92e2edee8"
format_stamp: "Formatted at 2026-08-06 21:11:45 on dist-test-slave-lgr5"
server_key: "b94d6915a68405df06954472fb1b1889"
server_key_iv: "079b5d9661790d79a5634d9b7c308b5f"
server_key_version: "encryptionkey@0"
I20260806 21:11:45.234665 21477 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:11:45.237181 21501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:45.238389 21477 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 21:11:45.238538 21477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "3687dcadd3964dd3ac2323f92e2edee8"
format_stamp: "Formatted at 2026-08-06 21:11:45 on dist-test-slave-lgr5"
server_key: "b94d6915a68405df06954472fb1b1889"
server_key_iv: "079b5d9661790d79a5634d9b7c308b5f"
server_key_version: "encryptionkey@0"
I20260806 21:11:45.238646 21477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:45.248879 21477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:45.252027 21477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:45.252240 21477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:45.259205 21477 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.62:39143
I20260806 21:11:45.259513 21579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.62:39143 every 8 connection(s)
I20260806 21:11:45.260501 21477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:11:45.263664 21580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:45.269685 21580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3687dcadd3964dd3ac2323f92e2edee8: Bootstrap starting.
I20260806 21:11:45.270530 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 21477
I20260806 21:11:45.270653 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:11:45.270948 17096 external_mini_cluster.cc:1428] Setting key 9367433f8cae2ff52cbf6e58d13132a3
I20260806 21:11:45.272778 21580 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3687dcadd3964dd3ac2323f92e2edee8: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:45.273526 21580 log.cc:824] T 00000000000000000000000000000000 P 3687dcadd3964dd3ac2323f92e2edee8: Log is configured to *not* fsync() on all Append() calls
I20260806 21:11:45.275579 21580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3687dcadd3964dd3ac2323f92e2edee8: No bootstrap required, opened a new log
Aug 06 21:11:45 dist-test-slave-lgr5 krb5kdc[21458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:11:45.280843 21580 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3687dcadd3964dd3ac2323f92e2edee8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3687dcadd3964dd3ac2323f92e2edee8" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 39143 } }
I20260806 21:11:45.281239 21580 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3687dcadd3964dd3ac2323f92e2edee8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:45.281335 21580 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3687dcadd3964dd3ac2323f92e2edee8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3687dcadd3964dd3ac2323f92e2edee8, State: Initialized, Role: FOLLOWER
I20260806 21:11:45.281922 21580 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3687dcadd3964dd3ac2323f92e2edee8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3687dcadd3964dd3ac2323f92e2edee8" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 39143 } }
I20260806 21:11:45.282158 21580 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3687dcadd3964dd3ac2323f92e2edee8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:11:45.282251 21580 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3687dcadd3964dd3ac2323f92e2edee8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:11:45.282388 21580 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3687dcadd3964dd3ac2323f92e2edee8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:45.283522 21580 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3687dcadd3964dd3ac2323f92e2edee8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3687dcadd3964dd3ac2323f92e2edee8" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 39143 } }
I20260806 21:11:45.284003 21580 leader_election.cc:302] T 00000000000000000000000000000000 P 3687dcadd3964dd3ac2323f92e2edee8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3687dcadd3964dd3ac2323f92e2edee8; no voters:
I20260806 21:11:45.284525 21580 leader_election.cc:288] T 00000000000000000000000000000000 P 3687dcadd3964dd3ac2323f92e2edee8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:11:45.284654 21586 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3687dcadd3964dd3ac2323f92e2edee8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:11:45.284855 21586 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3687dcadd3964dd3ac2323f92e2edee8 [term 1 LEADER]: Becoming Leader. State: Replica: 3687dcadd3964dd3ac2323f92e2edee8, State: Running, Role: LEADER
I20260806 21:11:45.285230 21586 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3687dcadd3964dd3ac2323f92e2edee8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3687dcadd3964dd3ac2323f92e2edee8" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 39143 } }
I20260806 21:11:45.286160 21580 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3687dcadd3964dd3ac2323f92e2edee8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:11:45.287182 21588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3687dcadd3964dd3ac2323f92e2edee8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3687dcadd3964dd3ac2323f92e2edee8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3687dcadd3964dd3ac2323f92e2edee8" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 39143 } } }
I20260806 21:11:45.287353 21588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3687dcadd3964dd3ac2323f92e2edee8 [sys.catalog]: This master's current role is: LEADER
I20260806 21:11:45.287600 21584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:45.272232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37432 (local address 127.16.178.62:39143)
0806 21:11:45.272739 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:45.272753 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:45.272790 (+ 37us) server_negotiation.cc:407] Connection header received
0806 21:11:45.273476 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:45.273504 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:45.273867 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:45.274191 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:45.275036 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:45.276048 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:45.277373 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:45.277890 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:45.278032 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:45.281382 (+ 3350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:45.281391 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:45.281414 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:45.281460 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:45.283500 (+ 2040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:45.284252 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:45.284260 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:45.284268 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:45.284450 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:45.284812 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:45.284815 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:45.284817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:45.285440 (+ 623us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:45.285670 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:45.286021 (+ 351us) server_negotiation.cc:299] Negotiation successful
0806 21:11:45.286515 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":152,"threads_started":1}
I20260806 21:11:45.287624 21588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3687dcadd3964dd3ac2323f92e2edee8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3687dcadd3964dd3ac2323f92e2edee8. Latest consensus state: current_term: 1 leader_uuid: "3687dcadd3964dd3ac2323f92e2edee8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3687dcadd3964dd3ac2323f92e2edee8" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 39143 } } }
I20260806 21:11:45.287715 21588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3687dcadd3964dd3ac2323f92e2edee8 [sys.catalog]: This master's current role is: LEADER
I20260806 21:11:45.291088 17096 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260806 21:11:45.291826 21612 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3687dcadd3964dd3ac2323f92e2edee8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 21:11:45.291913 21612 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 21:11:45.291982 21603 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:11:45.292744 21603 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:11:45.299371 21603 catalog_manager.cc:1347] Generated new cluster ID: 40d870ca1ee94c58bab757c2a683a24b
I20260806 21:11:45.299451 21603 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:11:45.312245 21603 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:11:45.313520 21603 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:11:45.317097 21584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:45.310972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37442 (local address 127.16.178.62:39143)
0806 21:11:45.311101 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:45.311106 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:45.311229 (+ 123us) server_negotiation.cc:407] Connection header received
0806 21:11:45.311423 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:45.311427 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:45.311482 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:45.311573 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:45.312375 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:45.313158 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:45.314433 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:45.314809 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:45.314905 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:45.315976 (+ 1071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:45.315980 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:45.315983 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:11:45.316591 (+ 608us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:45.316627 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:11:45.316646 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:45.316728 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:45.316856 (+ 128us) server_negotiation.cc:299] Negotiation successful
0806 21:11:45.316936 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 21:11:45.320394 21603 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3687dcadd3964dd3ac2323f92e2edee8: Generated new TSK 0
I20260806 21:11:45.321146 21603 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:11:45.330121 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 21477
2026-08-06T21:11:45Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1533 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:11:45 dist-test-slave-lgr5 krb5kdc[21629](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:11:45 dist-test-slave-lgr5 krb5kdc[21629](info): set up 2 sockets
Aug 06 21:11:45 dist-test-slave-lgr5 krb5kdc[21629](info): commencing operation
krb5kdc: starting...
W20260806 21:11:46.495357 17096 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Aug 06 21:11:46 dist-test-slave-lgr5 krb5kdc[21629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050706, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:11:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:11:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.178.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
I20260806 21:11:46.625237 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:37569
--webserver_interface=127.16.178.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:39117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:37569
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:11:46.739790 21646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:46.740098 21646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:46.740159 21646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:46.744204 21646 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:11:46.744272 21646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:46.744299 21646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:46.744320 21646 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:11:46.744349 21646 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:11:46.749701 21646 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:39117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:37569
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:37569
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.178.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:46.750829 21646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:46.751854 21646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:46.760897 21654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:46.760855 21658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:46.760855 21655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:46.761308 21646 server_base.cc:1034] running on GCE node
I20260806 21:11:46.761925 21646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:46.763134 21646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:46.764295 21646 hybrid_clock.cc:648] HybridClock initialized: now 1786050706764279 us; error 30 us; skew 500 ppm
Aug 06 21:11:46 dist-test-slave-lgr5 krb5kdc[21629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050706, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:46.767163 21646 init.cc:377] Logged in from keytab as kudu/127.16.178.62@KRBTEST.COM (short username kudu)
I20260806 21:11:46.768262 21646 webserver.cc:460] Webserver started at http://127.16.178.62:33501/ using document root <none> and password file <none>
I20260806 21:11:46.768839 21646 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:46.768909 21646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:46.769120 21646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:46.771241 21646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "ebdd889a8fcd468084a9bc88681eff87"
format_stamp: "Formatted at 2026-08-06 21:11:46 on dist-test-slave-lgr5"
server_key: "8c483cca75ae7666a37dc7a88a8cbf47"
server_key_iv: "57f0ad30ece57efcded9f54daf7621d7"
server_key_version: "encryptionkey@0"
I20260806 21:11:46.771800 21646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "ebdd889a8fcd468084a9bc88681eff87"
format_stamp: "Formatted at 2026-08-06 21:11:46 on dist-test-slave-lgr5"
server_key: "8c483cca75ae7666a37dc7a88a8cbf47"
server_key_iv: "57f0ad30ece57efcded9f54daf7621d7"
server_key_version: "encryptionkey@0"
I20260806 21:11:46.775552 21646 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260806 21:11:46.778020 21670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:46.779229 21646 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 21:11:46.779354 21646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "ebdd889a8fcd468084a9bc88681eff87"
format_stamp: "Formatted at 2026-08-06 21:11:46 on dist-test-slave-lgr5"
server_key: "8c483cca75ae7666a37dc7a88a8cbf47"
server_key_iv: "57f0ad30ece57efcded9f54daf7621d7"
server_key_version: "encryptionkey@0"
I20260806 21:11:46.779469 21646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:46.796837 21646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:46.800343 21646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:46.800523 21646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:46.807729 21646 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.62:37569
I20260806 21:11:46.807972 21756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.62:37569 every 8 connection(s)
I20260806 21:11:46.808693 21646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:11:46.811386 21757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:46.812273 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 21646
I20260806 21:11:46.812394 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:11:46.812652 17096 external_mini_cluster.cc:1428] Setting key a66216e05f845c4c8957ed82a0a6956d
I20260806 21:11:46.818231 21757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebdd889a8fcd468084a9bc88681eff87: Bootstrap starting.
I20260806 21:11:46.821506 21757 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ebdd889a8fcd468084a9bc88681eff87: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:46.822319 21757 log.cc:824] T 00000000000000000000000000000000 P ebdd889a8fcd468084a9bc88681eff87: Log is configured to *not* fsync() on all Append() calls
Aug 06 21:11:46 dist-test-slave-lgr5 krb5kdc[21629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050706, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:11:46.824581 21757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebdd889a8fcd468084a9bc88681eff87: No bootstrap required, opened a new log
I20260806 21:11:46.827648 21760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:46.814055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43172 (local address 127.16.178.62:37569)
0806 21:11:46.814475 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:46.814495 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:46.814543 (+ 48us) server_negotiation.cc:407] Connection header received
0806 21:11:46.815355 (+ 812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:46.815380 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:46.815784 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:46.816200 (+ 416us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:46.816902 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:46.818421 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:46.819657 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:46.820413 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:46.820572 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:46.823284 (+ 2712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:46.823289 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:46.823303 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:46.823361 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:46.824968 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:46.825449 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:46.825454 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:46.825459 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:46.825566 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:46.825833 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:46.825835 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:46.825836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:46.826202 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:46.826355 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:46.826629 (+ 274us) server_negotiation.cc:299] Negotiation successful
0806 21:11:46.826874 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":113,"threads_started":1}
I20260806 21:11:46.828508 21757 raft_consensus.cc:348] T 00000000000000000000000000000000 P ebdd889a8fcd468084a9bc88681eff87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebdd889a8fcd468084a9bc88681eff87" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 37569 } }
I20260806 21:11:46.828800 21757 raft_consensus.cc:374] T 00000000000000000000000000000000 P ebdd889a8fcd468084a9bc88681eff87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:46.828860 21757 raft_consensus.cc:729] T 00000000000000000000000000000000 P ebdd889a8fcd468084a9bc88681eff87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebdd889a8fcd468084a9bc88681eff87, State: Initialized, Role: FOLLOWER
I20260806 21:11:46.829326 21757 consensus_queue.cc:260] T 00000000000000000000000000000000 P ebdd889a8fcd468084a9bc88681eff87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebdd889a8fcd468084a9bc88681eff87" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 37569 } }
I20260806 21:11:46.829485 21757 raft_consensus.cc:388] T 00000000000000000000000000000000 P ebdd889a8fcd468084a9bc88681eff87 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:11:46.829561 21757 raft_consensus.cc:482] T 00000000000000000000000000000000 P ebdd889a8fcd468084a9bc88681eff87 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:11:46.829659 21757 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ebdd889a8fcd468084a9bc88681eff87 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:46.830542 21757 raft_consensus.cc:504] T 00000000000000000000000000000000 P ebdd889a8fcd468084a9bc88681eff87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebdd889a8fcd468084a9bc88681eff87" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 37569 } }
I20260806 21:11:46.830860 21757 leader_election.cc:302] T 00000000000000000000000000000000 P ebdd889a8fcd468084a9bc88681eff87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ebdd889a8fcd468084a9bc88681eff87; no voters:
I20260806 21:11:46.831146 21757 leader_election.cc:288] T 00000000000000000000000000000000 P ebdd889a8fcd468084a9bc88681eff87 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:11:46.831274 21763 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ebdd889a8fcd468084a9bc88681eff87 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:11:46.831506 21763 raft_consensus.cc:686] T 00000000000000000000000000000000 P ebdd889a8fcd468084a9bc88681eff87 [term 1 LEADER]: Becoming Leader. State: Replica: ebdd889a8fcd468084a9bc88681eff87, State: Running, Role: LEADER
I20260806 21:11:46.831807 21763 consensus_queue.cc:237] T 00000000000000000000000000000000 P ebdd889a8fcd468084a9bc88681eff87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebdd889a8fcd468084a9bc88681eff87" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 37569 } }
I20260806 21:11:46.832327 21757 sys_catalog.cc:564] T 00000000000000000000000000000000 P ebdd889a8fcd468084a9bc88681eff87 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:11:46.833446 21765 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebdd889a8fcd468084a9bc88681eff87 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ebdd889a8fcd468084a9bc88681eff87. Latest consensus state: current_term: 1 leader_uuid: "ebdd889a8fcd468084a9bc88681eff87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebdd889a8fcd468084a9bc88681eff87" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 37569 } } }
I20260806 21:11:46.833560 21765 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebdd889a8fcd468084a9bc88681eff87 [sys.catalog]: This master's current role is: LEADER
I20260806 21:11:46.833855 21764 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebdd889a8fcd468084a9bc88681eff87 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ebdd889a8fcd468084a9bc88681eff87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebdd889a8fcd468084a9bc88681eff87" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 37569 } } }
I20260806 21:11:46.833937 21775 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:11:46.834024 21764 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebdd889a8fcd468084a9bc88681eff87 [sys.catalog]: This master's current role is: LEADER
I20260806 21:11:46.836823 21775 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:11:46.842383 21775 catalog_manager.cc:1347] Generated new cluster ID: e256bbc013174050b87f448f1b757a8e
I20260806 21:11:46.842464 21775 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:11:46.852833 21775 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:11:46.853714 21775 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:11:46.858940 21775 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ebdd889a8fcd468084a9bc88681eff87: Generated new TSK 0
I20260806 21:11:46.859581 21775 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.178.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab.
Entry for principal kudu/127.16.178.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab.
I20260806 21:11:46.892632 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.1:0
--local_ip_for_outbound_sockets=127.16.178.1
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:37569
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab
--principal=kudu/127.16.178.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:39117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:11:47.009073 21791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:47.009387 21791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:47.009433 21791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:47.013552 21791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:47.013629 21791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:47.013689 21791 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.1
I20260806 21:11:47.019335 21791 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:39117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab
--principal=kudu/127.16.178.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:37569
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21791
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.1
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:47.020651 21791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:47.021744 21791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 21:11:47.031720 21791 server_base.cc:1034] running on GCE node
W20260806 21:11:47.031708 21799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:47.031749 21802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:47.031738 21798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:47.032434 21791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:47.033116 21791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:47.034296 21791 hybrid_clock.cc:648] HybridClock initialized: now 1786050707034267 us; error 34 us; skew 500 ppm
Aug 06 21:11:47 dist-test-slave-lgr5 krb5kdc[21629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050707, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:47.037405 21791 init.cc:377] Logged in from keytab as kudu/127.16.178.1@KRBTEST.COM (short username kudu)
I20260806 21:11:47.038381 21791 webserver.cc:460] Webserver started at http://127.16.178.1:40295/ using document root <none> and password file <none>
I20260806 21:11:47.038947 21791 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:47.039012 21791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:47.039284 21791 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:47.041549 21791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-0/data/instance:
uuid: "5a092cdd9d0b4d8084400081a1ecd6b8"
format_stamp: "Formatted at 2026-08-06 21:11:47 on dist-test-slave-lgr5"
server_key: "5a1df0dce2600eed2dc8365666280494"
server_key_iv: "e81a3dc6d9c37238c697284ecf06b738"
server_key_version: "encryptionkey@0"
I20260806 21:11:47.042136 21791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance:
uuid: "5a092cdd9d0b4d8084400081a1ecd6b8"
format_stamp: "Formatted at 2026-08-06 21:11:47 on dist-test-slave-lgr5"
server_key: "5a1df0dce2600eed2dc8365666280494"
server_key_iv: "e81a3dc6d9c37238c697284ecf06b738"
server_key_version: "encryptionkey@0"
I20260806 21:11:47.046353 21791 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:11:47.049369 21816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:47.050925 21791 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:11:47.051050 21791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-0/wal
uuid: "5a092cdd9d0b4d8084400081a1ecd6b8"
format_stamp: "Formatted at 2026-08-06 21:11:47 on dist-test-slave-lgr5"
server_key: "5a1df0dce2600eed2dc8365666280494"
server_key_iv: "e81a3dc6d9c37238c697284ecf06b738"
server_key_version: "encryptionkey@0"
I20260806 21:11:47.051182 21791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:47.064772 21791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:47.068190 21791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:47.068375 21791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:47.069001 21791 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:11:47.069850 21791 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:11:47.069906 21791 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:47.069974 21791 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:11:47.070003 21791 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:47.078886 21791 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.1:45651
I20260806 21:11:47.078917 21996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.1:45651 every 8 connection(s)
I20260806 21:11:47.079814 21791 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
I20260806 21:11:47.080546 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 21791
I20260806 21:11:47.080677 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance
I20260806 21:11:47.080992 17096 external_mini_cluster.cc:1428] Setting key 7037daf6c84a24c707e21c7c4c022ebe
Aug 06 21:11:47 dist-test-slave-lgr5 krb5kdc[21629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050707, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.178.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.2@KRBTEST.COM" created.
I20260806 21:11:47.096673 21760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:47.081916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:35927 (local address 127.16.178.62:37569)
0806 21:11:47.082040 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:47.082044 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:47.083258 (+ 1214us) server_negotiation.cc:407] Connection header received
0806 21:11:47.085211 (+ 1953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:47.085215 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:47.085275 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:47.085375 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:47.086802 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:47.087366 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:47.089285 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:47.089532 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:47.089622 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:47.092732 (+ 3110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:47.092735 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:47.092737 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:47.092767 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:47.094220 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:47.094843 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:47.094847 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:47.094849 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:47.094944 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:47.095368 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:47.095371 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:47.095373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:47.095611 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:47.095702 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:47.096351 (+ 649us) server_negotiation.cc:299] Negotiation successful
0806 21:11:47.096499 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 21:11:47.097559 22002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:47.082411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37569 (local address 127.16.178.1:35927)
0806 21:11:47.083085 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:47.083151 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:47.084935 (+ 1784us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:47.085581 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:47.085589 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:47.085986 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:47.086616 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:47.086638 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:47.087505 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:47.087509 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:47.089086 (+ 1577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:47.089103 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:47.089725 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:47.089731 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:47.089974 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:47.090635 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:47.090660 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:47.092520 (+ 1860us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:47.094357 (+ 1837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:47.094363 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:47.094378 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:47.094676 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:47.095049 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:47.095052 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:47.095053 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:47.095222 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:47.095730 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:47.095737 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:47.096035 (+ 298us) client_negotiation.cc:769] Sending connection context
0806 21:11:47.096254 (+ 219us) client_negotiation.cc:240] Negotiation successful
0806 21:11:47.096582 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":150,"threads_started":1}
I20260806 21:11:47.099012 21997 heartbeater.cc:347] Connected to a master server at 127.16.178.62:37569
I20260806 21:11:47.099282 21997 heartbeater.cc:464] Registering TS with master...
I20260806 21:11:47.099906 21997 heartbeater.cc:511] Master 127.16.178.62:37569 requested a full tablet report, sending...
I20260806 21:11:47.101752 21698 ts_manager.cc:194] Registered new tserver with Master: 5a092cdd9d0b4d8084400081a1ecd6b8 (127.16.178.1:45651)
I20260806 21:11:47.102958 21698 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.1@KRBTEST.COM'} at 127.16.178.1:35927
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab.
Entry for principal kudu/127.16.178.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab.
I20260806 21:11:47.113435 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.2:0
--local_ip_for_outbound_sockets=127.16.178.2
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:37569
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab
--principal=kudu/127.16.178.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:39117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:11:47.231046 22007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:47.231369 22007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:47.231444 22007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:47.235801 22007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:47.235889 22007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:47.235987 22007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.2
I20260806 21:11:47.241696 22007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:39117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab
--principal=kudu/127.16.178.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:37569
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.2
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:47.242923 22007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:47.244035 22007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:47.254333 22016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:47.254362 22019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:47.254335 22013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:47.254539 22007 server_base.cc:1034] running on GCE node
I20260806 21:11:47.255064 22007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:47.255774 22007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:47.256942 22007 hybrid_clock.cc:648] HybridClock initialized: now 1786050707256917 us; error 34 us; skew 500 ppm
Aug 06 21:11:47 dist-test-slave-lgr5 krb5kdc[21629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050707, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:47.259845 22007 init.cc:377] Logged in from keytab as kudu/127.16.178.2@KRBTEST.COM (short username kudu)
I20260806 21:11:47.260921 22007 webserver.cc:460] Webserver started at http://127.16.178.2:39537/ using document root <none> and password file <none>
I20260806 21:11:47.261519 22007 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:47.261584 22007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:47.261768 22007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:47.263859 22007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-1/data/instance:
uuid: "bb6df536e98545989d038d2e7e844efc"
format_stamp: "Formatted at 2026-08-06 21:11:47 on dist-test-slave-lgr5"
server_key: "621b389ab9d003d274c9d26040c73d6f"
server_key_iv: "a5271a33db9a9debdec43edb821fb6a2"
server_key_version: "encryptionkey@0"
I20260806 21:11:47.264482 22007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance:
uuid: "bb6df536e98545989d038d2e7e844efc"
format_stamp: "Formatted at 2026-08-06 21:11:47 on dist-test-slave-lgr5"
server_key: "621b389ab9d003d274c9d26040c73d6f"
server_key_iv: "a5271a33db9a9debdec43edb821fb6a2"
server_key_version: "encryptionkey@0"
I20260806 21:11:47.268273 22007 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 21:11:47.270807 22027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:47.272071 22007 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 21:11:47.272220 22007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-1/wal
uuid: "bb6df536e98545989d038d2e7e844efc"
format_stamp: "Formatted at 2026-08-06 21:11:47 on dist-test-slave-lgr5"
server_key: "621b389ab9d003d274c9d26040c73d6f"
server_key_iv: "a5271a33db9a9debdec43edb821fb6a2"
server_key_version: "encryptionkey@0"
I20260806 21:11:47.272330 22007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:47.283699 22007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:47.286957 22007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:47.287173 22007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:47.287815 22007 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:11:47.288744 22007 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:11:47.288793 22007 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:47.288837 22007 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:11:47.288866 22007 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:47.297663 22007 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.2:33547
I20260806 21:11:47.297685 22212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.2:33547 every 8 connection(s)
I20260806 21:11:47.298612 22007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
I20260806 21:11:47.301132 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 22007
I20260806 21:11:47.301276 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance
I20260806 21:11:47.301618 17096 external_mini_cluster.cc:1428] Setting key 483112b093fa29f85ee3f84a6aed1745
Aug 06 21:11:47 dist-test-slave-lgr5 krb5kdc[21629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050707, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.2@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.178.3@KRBTEST.COM; defaulting to no policy
I20260806 21:11:47.312629 21760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:47.300539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:43179 (local address 127.16.178.62:37569)
0806 21:11:47.300688 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:47.300692 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:47.301652 (+ 960us) server_negotiation.cc:407] Connection header received
0806 21:11:47.303117 (+ 1465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:47.303120 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:47.303172 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:47.303246 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:47.304479 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:47.305001 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:47.306127 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:47.306346 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:47.306435 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:47.308963 (+ 2528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:47.308966 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:47.308968 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:47.308993 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:47.310408 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:47.310947 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:47.310950 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:47.310952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:47.311050 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:47.311381 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:47.311385 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:47.311387 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:47.311605 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:47.311697 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:47.312294 (+ 597us) server_negotiation.cc:299] Negotiation successful
0806 21:11:47.312457 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 21:11:47.313539 22215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:47.300930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37569 (local address 127.16.178.2:43179)
0806 21:11:47.301507 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:47.301552 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:47.302912 (+ 1360us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:47.303440 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:47.303447 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:47.303807 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:47.304336 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:47.304360 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:47.305119 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:47.305122 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:47.306008 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:47.306017 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:47.306489 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:47.306493 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:47.306654 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:47.307258 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:47.307275 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:47.308792 (+ 1517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:47.310546 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:47.310553 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:47.310571 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:47.310846 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:47.311138 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:47.311141 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:47.311144 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:47.311292 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:47.311708 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:47.311716 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:47.312002 (+ 286us) client_negotiation.cc:769] Sending connection context
0806 21:11:47.312226 (+ 224us) client_negotiation.cc:240] Negotiation successful
0806 21:11:47.312623 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":145,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.3@KRBTEST.COM" created.
I20260806 21:11:47.314849 22213 heartbeater.cc:347] Connected to a master server at 127.16.178.62:37569
I20260806 21:11:47.315177 22213 heartbeater.cc:464] Registering TS with master...
I20260806 21:11:47.315858 22213 heartbeater.cc:511] Master 127.16.178.62:37569 requested a full tablet report, sending...
I20260806 21:11:47.317276 21698 ts_manager.cc:194] Registered new tserver with Master: bb6df536e98545989d038d2e7e844efc (127.16.178.2:33547)
I20260806 21:11:47.317960 21698 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.2@KRBTEST.COM'} at 127.16.178.2:43179
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab.
Entry for principal kudu/127.16.178.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab.
I20260806 21:11:47.332068 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.3:0
--local_ip_for_outbound_sockets=127.16.178.3
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:37569
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab
--principal=kudu/127.16.178.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:39117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:11:47.450284 22218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:47.450672 22218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:47.450747 22218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:47.455206 22218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:47.455267 22218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:47.455327 22218 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.3
I20260806 21:11:47.460637 22218 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:39117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab
--principal=kudu/127.16.178.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:37569
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22218
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.3
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:47.461615 22218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:47.462584 22218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:47.471947 22231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:47.471952 22227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:47.471952 22226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:47.472296 22218 server_base.cc:1034] running on GCE node
I20260806 21:11:47.472656 22218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:47.473272 22218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:47.474469 22218 hybrid_clock.cc:648] HybridClock initialized: now 1786050707474446 us; error 36 us; skew 500 ppm
Aug 06 21:11:47 dist-test-slave-lgr5 krb5kdc[21629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050707, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:47.477236 22218 init.cc:377] Logged in from keytab as kudu/127.16.178.3@KRBTEST.COM (short username kudu)
I20260806 21:11:47.478096 22218 webserver.cc:460] Webserver started at http://127.16.178.3:44955/ using document root <none> and password file <none>
I20260806 21:11:47.478648 22218 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:47.478717 22218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:47.478935 22218 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:47.481163 22218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-2/data/instance:
uuid: "ce190639c72d4038a3e47b8eb51511ed"
format_stamp: "Formatted at 2026-08-06 21:11:47 on dist-test-slave-lgr5"
server_key: "47b025df6673de0d5f24b03c9ba32720"
server_key_iv: "377cebe961caf8cdbe1344139afd366a"
server_key_version: "encryptionkey@0"
I20260806 21:11:47.481736 22218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance:
uuid: "ce190639c72d4038a3e47b8eb51511ed"
format_stamp: "Formatted at 2026-08-06 21:11:47 on dist-test-slave-lgr5"
server_key: "47b025df6673de0d5f24b03c9ba32720"
server_key_iv: "377cebe961caf8cdbe1344139afd366a"
server_key_version: "encryptionkey@0"
I20260806 21:11:47.485903 22218 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:11:47.488510 22239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:47.489742 22218 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 21:11:47.489877 22218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-2/wal
uuid: "ce190639c72d4038a3e47b8eb51511ed"
format_stamp: "Formatted at 2026-08-06 21:11:47 on dist-test-slave-lgr5"
server_key: "47b025df6673de0d5f24b03c9ba32720"
server_key_iv: "377cebe961caf8cdbe1344139afd366a"
server_key_version: "encryptionkey@0"
I20260806 21:11:47.489985 22218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:47.502754 22218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:47.505908 22218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:47.506111 22218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:47.506742 22218 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:11:47.507624 22218 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:11:47.507676 22218 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:47.507740 22218 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:11:47.507766 22218 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:47.516446 22218 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.3:40753
I20260806 21:11:47.516484 22430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.3:40753 every 8 connection(s)
I20260806 21:11:47.517372 22218 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
I20260806 21:11:47.520231 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 22218
I20260806 21:11:47.520370 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance
I20260806 21:11:47.520704 17096 external_mini_cluster.cc:1428] Setting key 6d9a0ff54c59f427750e9a16b1890d0a
Aug 06 21:11:47 dist-test-slave-lgr5 krb5kdc[21629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050707, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.3@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:11:47.530441 21760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:47.519178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:36337 (local address 127.16.178.62:37569)
0806 21:11:47.519331 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:47.519335 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:47.520200 (+ 865us) server_negotiation.cc:407] Connection header received
0806 21:11:47.521787 (+ 1587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:47.521789 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:47.521840 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:47.521895 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:47.523173 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:47.523700 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:47.524898 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:47.525141 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:47.525231 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:47.527546 (+ 2315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:47.527549 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:47.527551 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:47.527576 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:47.528771 (+ 1195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:47.529217 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:47.529220 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:47.529221 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:47.529286 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:47.529540 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:47.529542 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:47.529543 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:47.529714 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:47.529784 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:47.530226 (+ 442us) server_negotiation.cc:299] Negotiation successful
0806 21:11:47.530318 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 21:11:47.531332 22435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:47.519541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37569 (local address 127.16.178.3:36337)
0806 21:11:47.520009 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:47.520068 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:47.521589 (+ 1521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:47.522068 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:47.522077 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:47.522462 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:47.523048 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:47.523064 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:47.523799 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:47.523801 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:47.524714 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:47.524724 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:47.525292 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:47.525296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:47.525447 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:47.525965 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:47.525981 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:47.527385 (+ 1404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:47.528878 (+ 1493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:47.528883 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:47.528896 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:47.529144 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:47.529344 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:47.529346 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:47.529347 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:47.529478 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:47.529800 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:47.529805 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:47.530042 (+ 237us) client_negotiation.cc:769] Sending connection context
0806 21:11:47.530196 (+ 154us) client_negotiation.cc:240] Negotiation successful
0806 21:11:47.530473 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":104,"threads_started":1}
I20260806 21:11:47.532495 22431 heartbeater.cc:347] Connected to a master server at 127.16.178.62:37569
I20260806 21:11:47.532761 22431 heartbeater.cc:464] Registering TS with master...
I20260806 21:11:47.533264 22431 heartbeater.cc:511] Master 127.16.178.62:37569 requested a full tablet report, sending...
I20260806 21:11:47.534477 21698 ts_manager.cc:194] Registered new tserver with Master: ce190639c72d4038a3e47b8eb51511ed (127.16.178.3:40753)
I20260806 21:11:47.535077 21698 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.3@KRBTEST.COM'} at 127.16.178.3:36337
I20260806 21:11:47.546799 17096 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 21:11:47 dist-test-slave-lgr5 krb5kdc[21629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050707, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 06 21:11:47 dist-test-slave-lgr5 krb5kdc[21629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050707, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:11:47.573968 21760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:47.564933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43180 (local address 127.16.178.62:37569)
0806 21:11:47.565084 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:47.565089 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:47.565252 (+ 163us) server_negotiation.cc:407] Connection header received
0806 21:11:47.565410 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:47.565412 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:47.565462 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:47.565534 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:47.566272 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:47.566795 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:47.567720 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:47.567934 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:47.568030 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:47.570646 (+ 2616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:47.570649 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:47.570651 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:47.570684 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:47.572010 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:47.572488 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:47.572492 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:47.572494 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:47.572582 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:47.572848 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:47.572851 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:47.572853 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:47.573116 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:47.573256 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:47.573597 (+ 341us) server_negotiation.cc:299] Negotiation successful
0806 21:11:47.573746 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260806 21:11:47.574462 21698 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:43180
I20260806 21:11:47.575512 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 21791
I20260806 21:11:47.591179 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 22007
I20260806 21:11:47.602679 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 22218
I20260806 21:11:47.615501 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 21646
2026-08-06T21:11:47Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2276 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:11:47 dist-test-slave-lgr5 krb5kdc[22449](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:11:47 dist-test-slave-lgr5 krb5kdc[22449](info): set up 2 sockets
Aug 06 21:11:47 dist-test-slave-lgr5 krb5kdc[22449](info): commencing operation
krb5kdc: starting...
W20260806 21:11:48.764055 17096 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Aug 06 21:11:48 dist-test-slave-lgr5 krb5kdc[22449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:11:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:11:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.178.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
I20260806 21:11:48.895833 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:45621
--webserver_interface=127.16.178.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:46729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:45621
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:11:49.017542 22466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:49.017874 22466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:49.017933 22466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:49.022177 22466 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:11:49.022255 22466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:49.022281 22466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:49.022313 22466 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:11:49.022341 22466 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:11:49.027936 22466 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:46729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:45621
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:45621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.178.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:49.029086 22466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:49.030126 22466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 21:11:49.038805 22466 server_base.cc:1034] running on GCE node
W20260806 21:11:49.038820 22474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:49.038832 22473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:49.038837 22478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:49.039649 22466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:49.040846 22466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:49.042026 22466 hybrid_clock.cc:648] HybridClock initialized: now 1786050709042002 us; error 39 us; skew 500 ppm
Aug 06 21:11:49 dist-test-slave-lgr5 krb5kdc[22449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050709, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:49.048978 22466 init.cc:377] Logged in from keytab as kudu/127.16.178.62@KRBTEST.COM (short username kudu)
I20260806 21:11:49.050047 22466 webserver.cc:460] Webserver started at http://127.16.178.62:39947/ using document root <none> and password file <none>
I20260806 21:11:49.050640 22466 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:49.050714 22466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:49.050944 22466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:49.053224 22466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "af3927346f2d482bb54ae88f8efefdfa"
format_stamp: "Formatted at 2026-08-06 21:11:49 on dist-test-slave-lgr5"
server_key: "a0ad0f03c5c5ebed1fa3feda37d74856"
server_key_iv: "586076764bd09a04fbd337807088e1a9"
server_key_version: "encryptionkey@0"
I20260806 21:11:49.053793 22466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "af3927346f2d482bb54ae88f8efefdfa"
format_stamp: "Formatted at 2026-08-06 21:11:49 on dist-test-slave-lgr5"
server_key: "a0ad0f03c5c5ebed1fa3feda37d74856"
server_key_iv: "586076764bd09a04fbd337807088e1a9"
server_key_version: "encryptionkey@0"
I20260806 21:11:49.057827 22466 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:11:49.060562 22488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:49.061856 22466 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:11:49.062016 22466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "af3927346f2d482bb54ae88f8efefdfa"
format_stamp: "Formatted at 2026-08-06 21:11:49 on dist-test-slave-lgr5"
server_key: "a0ad0f03c5c5ebed1fa3feda37d74856"
server_key_iv: "586076764bd09a04fbd337807088e1a9"
server_key_version: "encryptionkey@0"
I20260806 21:11:49.062130 22466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:49.075376 22466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:49.079054 22466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:49.079281 22466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:49.088421 22466 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.62:45621
I20260806 21:11:49.088439 22566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.62:45621 every 8 connection(s)
I20260806 21:11:49.089629 22466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:11:49.093230 22569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:49.094043 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 22466
I20260806 21:11:49.094168 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:11:49.094450 17096 external_mini_cluster.cc:1428] Setting key 8a872529efefc1c73589d4f01dfd627c
I20260806 21:11:49.100353 22569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af3927346f2d482bb54ae88f8efefdfa: Bootstrap starting.
I20260806 21:11:49.103266 22569 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af3927346f2d482bb54ae88f8efefdfa: Neither blocks nor log segments found. Creating new log.
Aug 06 21:11:49 dist-test-slave-lgr5 krb5kdc[22449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:11:49.104177 22569 log.cc:824] T 00000000000000000000000000000000 P af3927346f2d482bb54ae88f8efefdfa: Log is configured to *not* fsync() on all Append() calls
I20260806 21:11:49.106374 22569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af3927346f2d482bb54ae88f8efefdfa: No bootstrap required, opened a new log
I20260806 21:11:49.110334 22576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:49.095967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42006 (local address 127.16.178.62:45621)
0806 21:11:49.096442 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:49.096458 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:49.096497 (+ 39us) server_negotiation.cc:407] Connection header received
0806 21:11:49.097182 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:49.097202 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:49.097562 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:49.097873 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:49.098888 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:49.100001 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:49.101195 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:49.101733 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:49.101892 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:49.104872 (+ 2980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:49.104878 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:49.104896 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:49.104931 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:49.106826 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:49.107354 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:49.107360 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:49.107366 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:49.107511 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:49.107830 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:49.107833 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:49.107834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:49.108399 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:49.108617 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:49.108965 (+ 348us) server_negotiation.cc:299] Negotiation successful
0806 21:11:49.109319 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":119,"threads_started":1}
I20260806 21:11:49.110262 22569 raft_consensus.cc:348] T 00000000000000000000000000000000 P af3927346f2d482bb54ae88f8efefdfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af3927346f2d482bb54ae88f8efefdfa" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 45621 } }
I20260806 21:11:49.110544 22569 raft_consensus.cc:374] T 00000000000000000000000000000000 P af3927346f2d482bb54ae88f8efefdfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:49.110616 22569 raft_consensus.cc:729] T 00000000000000000000000000000000 P af3927346f2d482bb54ae88f8efefdfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af3927346f2d482bb54ae88f8efefdfa, State: Initialized, Role: FOLLOWER
I20260806 21:11:49.111104 22569 consensus_queue.cc:260] T 00000000000000000000000000000000 P af3927346f2d482bb54ae88f8efefdfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af3927346f2d482bb54ae88f8efefdfa" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 45621 } }
I20260806 21:11:49.111256 22569 raft_consensus.cc:388] T 00000000000000000000000000000000 P af3927346f2d482bb54ae88f8efefdfa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:11:49.111349 22569 raft_consensus.cc:482] T 00000000000000000000000000000000 P af3927346f2d482bb54ae88f8efefdfa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:11:49.112172 22569 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af3927346f2d482bb54ae88f8efefdfa [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:49.113510 22569 raft_consensus.cc:504] T 00000000000000000000000000000000 P af3927346f2d482bb54ae88f8efefdfa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af3927346f2d482bb54ae88f8efefdfa" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 45621 } }
I20260806 21:11:49.113911 22569 leader_election.cc:302] T 00000000000000000000000000000000 P af3927346f2d482bb54ae88f8efefdfa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: af3927346f2d482bb54ae88f8efefdfa; no voters:
I20260806 21:11:49.114279 22569 leader_election.cc:288] T 00000000000000000000000000000000 P af3927346f2d482bb54ae88f8efefdfa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:11:49.114403 22580 raft_consensus.cc:2781] T 00000000000000000000000000000000 P af3927346f2d482bb54ae88f8efefdfa [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:11:49.114646 22580 raft_consensus.cc:686] T 00000000000000000000000000000000 P af3927346f2d482bb54ae88f8efefdfa [term 1 LEADER]: Becoming Leader. State: Replica: af3927346f2d482bb54ae88f8efefdfa, State: Running, Role: LEADER
I20260806 21:11:49.115090 22580 consensus_queue.cc:237] T 00000000000000000000000000000000 P af3927346f2d482bb54ae88f8efefdfa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af3927346f2d482bb54ae88f8efefdfa" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 45621 } }
I20260806 21:11:49.115700 22569 sys_catalog.cc:564] T 00000000000000000000000000000000 P af3927346f2d482bb54ae88f8efefdfa [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:11:49.116982 22586 sys_catalog.cc:455] T 00000000000000000000000000000000 P af3927346f2d482bb54ae88f8efefdfa [sys.catalog]: SysCatalogTable state changed. Reason: New leader af3927346f2d482bb54ae88f8efefdfa. Latest consensus state: current_term: 1 leader_uuid: "af3927346f2d482bb54ae88f8efefdfa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af3927346f2d482bb54ae88f8efefdfa" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 45621 } } }
I20260806 21:11:49.117183 22586 sys_catalog.cc:458] T 00000000000000000000000000000000 P af3927346f2d482bb54ae88f8efefdfa [sys.catalog]: This master's current role is: LEADER
I20260806 21:11:49.117630 22584 sys_catalog.cc:455] T 00000000000000000000000000000000 P af3927346f2d482bb54ae88f8efefdfa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "af3927346f2d482bb54ae88f8efefdfa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af3927346f2d482bb54ae88f8efefdfa" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 45621 } } }
I20260806 21:11:49.117827 22584 sys_catalog.cc:458] T 00000000000000000000000000000000 P af3927346f2d482bb54ae88f8efefdfa [sys.catalog]: This master's current role is: LEADER
W20260806 21:11:49.121345 22610 catalog_manager.cc:1558] T 00000000000000000000000000000000 P af3927346f2d482bb54ae88f8efefdfa: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 21:11:49.121434 22610 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 21:11:49.121522 22598 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:11:49.122283 22598 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:11:49.128396 22598 catalog_manager.cc:1347] Generated new cluster ID: b08e2e6d5bbb4381beba593835652354
I20260806 21:11:49.128500 22598 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:11:49.152134 22598 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:11:49.153632 22598 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:11:49.169795 22598 catalog_manager.cc:5868] T 00000000000000000000000000000000 P af3927346f2d482bb54ae88f8efefdfa: Generated new TSK 0
I20260806 21:11:49.170872 22598 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.178.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab.
Entry for principal kudu/127.16.178.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab.
I20260806 21:11:49.199517 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.1:0
--local_ip_for_outbound_sockets=127.16.178.1
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:45621
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab
--principal=kudu/127.16.178.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:46729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:11:49.313488 22619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:49.313758 22619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:49.313802 22619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:49.317932 22619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:49.317997 22619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:49.318054 22619 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.1
I20260806 21:11:49.323354 22619 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:46729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab
--principal=kudu/127.16.178.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:45621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22619
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.1
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:49.324488 22619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:49.325522 22619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:49.335994 22630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:49.336107 22619 server_base.cc:1034] running on GCE node
W20260806 21:11:49.336009 22627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:49.336051 22625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:49.336686 22619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:49.337486 22619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:49.338689 22619 hybrid_clock.cc:648] HybridClock initialized: now 1786050709338664 us; error 41 us; skew 500 ppm
Aug 06 21:11:49 dist-test-slave-lgr5 krb5kdc[22449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050709, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:49.341778 22619 init.cc:377] Logged in from keytab as kudu/127.16.178.1@KRBTEST.COM (short username kudu)
I20260806 21:11:49.342795 22619 webserver.cc:460] Webserver started at http://127.16.178.1:38743/ using document root <none> and password file <none>
I20260806 21:11:49.343393 22619 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:49.343463 22619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:49.343678 22619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:49.345913 22619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-0/data/instance:
uuid: "9c797678df524fd19d8e16d47b8f091f"
format_stamp: "Formatted at 2026-08-06 21:11:49 on dist-test-slave-lgr5"
server_key: "1cf935458791814b89d70cabc1e1def5"
server_key_iv: "3af5ae2c60ea133f2bbac347c8a5f275"
server_key_version: "encryptionkey@0"
I20260806 21:11:49.346505 22619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance:
uuid: "9c797678df524fd19d8e16d47b8f091f"
format_stamp: "Formatted at 2026-08-06 21:11:49 on dist-test-slave-lgr5"
server_key: "1cf935458791814b89d70cabc1e1def5"
server_key_iv: "3af5ae2c60ea133f2bbac347c8a5f275"
server_key_version: "encryptionkey@0"
I20260806 21:11:49.350874 22619 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260806 21:11:49.353820 22642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:49.355316 22619 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 21:11:49.355471 22619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-0/wal
uuid: "9c797678df524fd19d8e16d47b8f091f"
format_stamp: "Formatted at 2026-08-06 21:11:49 on dist-test-slave-lgr5"
server_key: "1cf935458791814b89d70cabc1e1def5"
server_key_iv: "3af5ae2c60ea133f2bbac347c8a5f275"
server_key_version: "encryptionkey@0"
I20260806 21:11:49.355604 22619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:49.376346 22619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:49.380749 22619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:49.380996 22619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:49.381793 22619 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:11:49.382897 22619 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:11:49.382964 22619 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:49.383046 22619 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:11:49.383077 22619 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:49.393734 22619 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.1:37733
I20260806 21:11:49.393781 22827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.1:37733 every 8 connection(s)
I20260806 21:11:49.394713 22619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
I20260806 21:11:49.397238 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 22619
I20260806 21:11:49.397373 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance
I20260806 21:11:49.397718 17096 external_mini_cluster.cc:1428] Setting key 36d31f6fadbbab61a3fd2681ebcbf4df
Aug 06 21:11:49 dist-test-slave-lgr5 krb5kdc[22449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050709, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.178.2@KRBTEST.COM; defaulting to no policy
I20260806 21:11:49.410331 22576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:49.396709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:60545 (local address 127.16.178.62:45621)
0806 21:11:49.396871 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:49.396875 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:49.397942 (+ 1067us) server_negotiation.cc:407] Connection header received
0806 21:11:49.399503 (+ 1561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:49.399507 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:49.399562 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:49.399634 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:49.400923 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:49.401687 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:49.403094 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:49.403427 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:49.403535 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:49.406439 (+ 2904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:49.406443 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:49.406446 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:49.406481 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:49.408114 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:49.408671 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:49.408674 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:49.408675 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:49.408756 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:49.409130 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:49.409135 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:49.409137 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:49.409372 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:49.409463 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:49.410030 (+ 567us) server_negotiation.cc:299] Negotiation successful
0806 21:11:49.410160 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.2@KRBTEST.COM" created.
I20260806 21:11:49.411504 22834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:49.397184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:45621 (local address 127.16.178.1:60545)
0806 21:11:49.397760 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:49.397825 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:49.399273 (+ 1448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:49.399825 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:49.399832 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:49.400251 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:49.400768 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:49.400782 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:49.401849 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:49.401852 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:49.402946 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:49.402959 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:49.403632 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:49.403635 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:49.403843 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:49.404605 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:49.404622 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:49.406227 (+ 1605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:49.408251 (+ 2024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:49.408259 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:49.408278 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:49.408569 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:49.408850 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:49.408853 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:49.408854 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:49.409007 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:49.409480 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:49.409486 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:49.409768 (+ 282us) client_negotiation.cc:769] Sending connection context
0806 21:11:49.409960 (+ 192us) client_negotiation.cc:240] Negotiation successful
0806 21:11:49.410332 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":175,"threads_started":1}
I20260806 21:11:49.412900 22829 heartbeater.cc:347] Connected to a master server at 127.16.178.62:45621
I20260806 21:11:49.413178 22829 heartbeater.cc:464] Registering TS with master...
I20260806 21:11:49.413830 22829 heartbeater.cc:511] Master 127.16.178.62:45621 requested a full tablet report, sending...
I20260806 21:11:49.415863 22515 ts_manager.cc:194] Registered new tserver with Master: 9c797678df524fd19d8e16d47b8f091f (127.16.178.1:37733)
I20260806 21:11:49.417827 22515 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.1@KRBTEST.COM'} at 127.16.178.1:60545
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab.
Entry for principal kudu/127.16.178.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab.
I20260806 21:11:49.431265 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.2:0
--local_ip_for_outbound_sockets=127.16.178.2
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:45621
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab
--principal=kudu/127.16.178.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:46729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:11:49.549192 22838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:49.549460 22838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:49.549503 22838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:49.553552 22838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:49.553635 22838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:49.553697 22838 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.2
I20260806 21:11:49.559106 22838 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:46729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab
--principal=kudu/127.16.178.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:45621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22838
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.2
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:49.560230 22838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:49.561268 22838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:49.570776 22847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:49.570919 22838 server_base.cc:1034] running on GCE node
W20260806 21:11:49.570756 22845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:49.570745 22851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:49.571399 22838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:49.572031 22838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:49.573236 22838 hybrid_clock.cc:648] HybridClock initialized: now 1786050709573210 us; error 37 us; skew 500 ppm
Aug 06 21:11:49 dist-test-slave-lgr5 krb5kdc[22449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050709, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:49.576007 22838 init.cc:377] Logged in from keytab as kudu/127.16.178.2@KRBTEST.COM (short username kudu)
I20260806 21:11:49.576879 22838 webserver.cc:460] Webserver started at http://127.16.178.2:39625/ using document root <none> and password file <none>
I20260806 21:11:49.577404 22838 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:49.577471 22838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:49.577665 22838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:49.579756 22838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-1/data/instance:
uuid: "ac8acb52e22c47f9a8c7ed0ab470f608"
format_stamp: "Formatted at 2026-08-06 21:11:49 on dist-test-slave-lgr5"
server_key: "de7dba813d6819cad2afd34d575c1eb9"
server_key_iv: "9adb927ea19a13225f8b23766d0077c3"
server_key_version: "encryptionkey@0"
I20260806 21:11:49.580348 22838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance:
uuid: "ac8acb52e22c47f9a8c7ed0ab470f608"
format_stamp: "Formatted at 2026-08-06 21:11:49 on dist-test-slave-lgr5"
server_key: "de7dba813d6819cad2afd34d575c1eb9"
server_key_iv: "9adb927ea19a13225f8b23766d0077c3"
server_key_version: "encryptionkey@0"
I20260806 21:11:49.584131 22838 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 21:11:49.586495 22862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:49.587641 22838 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 21:11:49.587752 22838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-1/wal
uuid: "ac8acb52e22c47f9a8c7ed0ab470f608"
format_stamp: "Formatted at 2026-08-06 21:11:49 on dist-test-slave-lgr5"
server_key: "de7dba813d6819cad2afd34d575c1eb9"
server_key_iv: "9adb927ea19a13225f8b23766d0077c3"
server_key_version: "encryptionkey@0"
I20260806 21:11:49.587864 22838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:49.601913 22838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:49.605161 22838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:49.605360 22838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:49.605983 22838 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:11:49.606874 22838 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:11:49.606923 22838 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:49.606979 22838 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:11:49.607007 22838 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:49.616129 22838 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.2:44717
I20260806 21:11:49.616166 23058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.2:44717 every 8 connection(s)
I20260806 21:11:49.617341 22838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
I20260806 21:11:49.619089 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 22838
I20260806 21:11:49.619194 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance
I20260806 21:11:49.619539 17096 external_mini_cluster.cc:1428] Setting key f45790ab174233e0f885f9677d763493
Aug 06 21:11:49 dist-test-slave-lgr5 krb5kdc[22449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050709, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.2@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.178.3@KRBTEST.COM; defaulting to no policy
I20260806 21:11:49.632030 22576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:49.619413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:40129 (local address 127.16.178.62:45621)
0806 21:11:49.619566 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:49.619571 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:49.620779 (+ 1208us) server_negotiation.cc:407] Connection header received
0806 21:11:49.622564 (+ 1785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:49.622567 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:49.622617 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:49.622686 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:49.624204 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:49.624938 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:49.626062 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:49.626283 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:49.626369 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:49.628940 (+ 2571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:49.628943 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:49.628945 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:49.628969 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:49.630292 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:49.630749 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:49.630752 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:49.630753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:49.630834 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:49.631126 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:49.631128 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:49.631129 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:49.631307 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:49.631377 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:49.631802 (+ 425us) server_negotiation.cc:299] Negotiation successful
0806 21:11:49.631895 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.3@KRBTEST.COM" created.
I20260806 21:11:49.633025 23061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:49.619894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:45621 (local address 127.16.178.2:40129)
0806 21:11:49.620584 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:49.620650 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:49.622286 (+ 1636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:49.622902 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:49.622912 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:49.623432 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:49.623992 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:49.624008 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:49.625072 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:49.625075 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:49.625933 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:49.625943 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:49.626426 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:49.626430 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:49.626594 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:49.627147 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:49.627163 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:49.628754 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:49.630405 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:49.630410 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:49.630422 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:49.630668 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:49.630914 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:49.630916 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:49.630917 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:49.631048 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:49.631376 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:49.631381 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:49.631614 (+ 233us) client_negotiation.cc:769] Sending connection context
0806 21:11:49.631761 (+ 147us) client_negotiation.cc:240] Negotiation successful
0806 21:11:49.632053 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":189,"threads_started":1}
I20260806 21:11:49.634281 23059 heartbeater.cc:347] Connected to a master server at 127.16.178.62:45621
I20260806 21:11:49.634557 23059 heartbeater.cc:464] Registering TS with master...
I20260806 21:11:49.635078 23059 heartbeater.cc:511] Master 127.16.178.62:45621 requested a full tablet report, sending...
I20260806 21:11:49.636166 22515 ts_manager.cc:194] Registered new tserver with Master: ac8acb52e22c47f9a8c7ed0ab470f608 (127.16.178.2:44717)
I20260806 21:11:49.636734 22515 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.2@KRBTEST.COM'} at 127.16.178.2:40129
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab.
Entry for principal kudu/127.16.178.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab.
I20260806 21:11:49.649037 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.3:0
--local_ip_for_outbound_sockets=127.16.178.3
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:45621
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab
--principal=kudu/127.16.178.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:46729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:11:49.765031 23067 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:49.765326 23067 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:49.765390 23067 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:49.769831 23067 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:49.769912 23067 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:49.769992 23067 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.3
I20260806 21:11:49.775616 23067 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:46729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab
--principal=kudu/127.16.178.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:45621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23067
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.3
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:49.776818 23067 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:49.777890 23067 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:49.787832 23077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:49.787890 23074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:49.787874 23067 server_base.cc:1034] running on GCE node
W20260806 21:11:49.787859 23075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:49.788631 23067 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:49.789305 23067 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:49.790494 23067 hybrid_clock.cc:648] HybridClock initialized: now 1786050709790464 us; error 44 us; skew 500 ppm
Aug 06 21:11:49 dist-test-slave-lgr5 krb5kdc[22449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050709, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:49.793741 23067 init.cc:377] Logged in from keytab as kudu/127.16.178.3@KRBTEST.COM (short username kudu)
I20260806 21:11:49.794906 23067 webserver.cc:460] Webserver started at http://127.16.178.3:46659/ using document root <none> and password file <none>
I20260806 21:11:49.795560 23067 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:49.795648 23067 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:49.795903 23067 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:49.798225 23067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-2/data/instance:
uuid: "01e07f666dad488a85a22d3aee9cf415"
format_stamp: "Formatted at 2026-08-06 21:11:49 on dist-test-slave-lgr5"
server_key: "7443dba899d1aa2d909dbecc73333ee2"
server_key_iv: "8e367fa9bec6f1d193ff6d75e65d3155"
server_key_version: "encryptionkey@0"
I20260806 21:11:49.798854 23067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance:
uuid: "01e07f666dad488a85a22d3aee9cf415"
format_stamp: "Formatted at 2026-08-06 21:11:49 on dist-test-slave-lgr5"
server_key: "7443dba899d1aa2d909dbecc73333ee2"
server_key_iv: "8e367fa9bec6f1d193ff6d75e65d3155"
server_key_version: "encryptionkey@0"
I20260806 21:11:49.802922 23067 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 21:11:49.805712 23090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:49.807129 23067 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 21:11:49.807288 23067 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-2/wal
uuid: "01e07f666dad488a85a22d3aee9cf415"
format_stamp: "Formatted at 2026-08-06 21:11:49 on dist-test-slave-lgr5"
server_key: "7443dba899d1aa2d909dbecc73333ee2"
server_key_iv: "8e367fa9bec6f1d193ff6d75e65d3155"
server_key_version: "encryptionkey@0"
I20260806 21:11:49.807427 23067 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:49.841429 23067 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:49.844619 23067 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:49.844848 23067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:49.845505 23067 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:11:49.846449 23067 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:11:49.846505 23067 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:49.846566 23067 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:11:49.846599 23067 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:49.856307 23067 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.3:36023
I20260806 21:11:49.856353 23284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.3:36023 every 8 connection(s)
I20260806 21:11:49.857324 23067 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
Aug 06 21:11:49 dist-test-slave-lgr5 krb5kdc[22449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050709, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.3@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:11:49.866896 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 23067
I20260806 21:11:49.867033 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance
I20260806 21:11:49.867305 17096 external_mini_cluster.cc:1428] Setting key 5e69f182b3fb8007bab794e6591914c8
I20260806 21:11:49.871269 22576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:49.859257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:43177 (local address 127.16.178.62:45621)
0806 21:11:49.859456 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:49.859462 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:49.860396 (+ 934us) server_negotiation.cc:407] Connection header received
0806 21:11:49.861755 (+ 1359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:49.861757 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:49.861807 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:49.861891 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:49.863048 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:49.863601 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:49.864682 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:49.864897 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:49.864977 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:49.867431 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:49.867435 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:49.867438 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:49.867468 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:49.869195 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:49.869702 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:49.869706 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:49.869708 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:49.869803 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:49.870145 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:49.870148 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:49.870149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:49.870372 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:49.870450 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:49.871028 (+ 578us) server_negotiation.cc:299] Negotiation successful
0806 21:11:49.871138 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 21:11:49.872248 23288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:49.859682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:45621 (local address 127.16.178.3:43177)
0806 21:11:49.860248 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:49.860291 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:49.861541 (+ 1250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:49.862042 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:49.862049 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:49.862412 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:49.862906 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:49.862921 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:49.863691 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:49.863693 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:49.864576 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:49.864585 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:49.865015 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:49.865018 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:49.865148 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:49.865758 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:49.865774 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:49.867241 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:49.869318 (+ 2077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:49.869323 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:49.869336 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:49.869611 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:49.869903 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:49.869906 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:49.869908 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:49.870060 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:49.870514 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:49.870520 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:49.870797 (+ 277us) client_negotiation.cc:769] Sending connection context
0806 21:11:49.870975 (+ 178us) client_negotiation.cc:240] Negotiation successful
0806 21:11:49.871260 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":144,"threads_started":1}
I20260806 21:11:49.873572 23285 heartbeater.cc:347] Connected to a master server at 127.16.178.62:45621
I20260806 21:11:49.873864 23285 heartbeater.cc:464] Registering TS with master...
I20260806 21:11:49.874428 23285 heartbeater.cc:511] Master 127.16.178.62:45621 requested a full tablet report, sending...
I20260806 21:11:49.875607 22515 ts_manager.cc:194] Registered new tserver with Master: 01e07f666dad488a85a22d3aee9cf415 (127.16.178.3:36023)
I20260806 21:11:49.876232 22515 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.3@KRBTEST.COM'} at 127.16.178.3:43177
I20260806 21:11:49.881848 17096 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 21:11:49 dist-test-slave-lgr5 krb5kdc[22449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050709, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 21:11:49 dist-test-slave-lgr5 krb5kdc[22449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050709, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.178.1@KRBTEST.COM
I20260806 21:11:49.908445 23298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:49.899331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33220 (local address 127.16.178.1:37733)
0806 21:11:49.899649 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:49.899667 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:49.899692 (+ 25us) server_negotiation.cc:407] Connection header received
0806 21:11:49.899747 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:49.899753 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:49.899964 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:49.900138 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:49.900933 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:49.901727 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:49.902554 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:49.902937 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:49.903017 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:49.905391 (+ 2374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:49.905406 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:49.905424 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:49.905450 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:49.906918 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:49.907292 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:49.907297 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:49.907298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:49.907372 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:49.907565 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:49.907566 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:49.907567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:49.907804 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:49.907971 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:49.908141 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 21:11:49.908276 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":120,"threads_started":1}
Aug 06 21:11:49 dist-test-slave-lgr5 krb5kdc[22449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050709, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:11:49.919044 22576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:49.911375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42080 (local address 127.16.178.62:45621)
0806 21:11:49.911520 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:49.911524 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:49.911538 (+ 14us) server_negotiation.cc:407] Connection header received
0806 21:11:49.911632 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:49.911634 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:49.911675 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:49.911737 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:49.912398 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:49.912910 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:49.913637 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:49.913833 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:49.913900 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:49.916140 (+ 2240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:49.916143 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:49.916145 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:49.916172 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:49.917542 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:49.917967 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:49.917969 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:49.917971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:49.918037 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:49.918267 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:49.918269 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:49.918270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:49.918501 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:49.918601 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:49.918773 (+ 172us) server_negotiation.cc:299] Negotiation successful
0806 21:11:49.918868 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260806 21:11:49.919479 22515 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42080
I20260806 21:11:49.920136 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 22619
I20260806 21:11:49.936570 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 22838
I20260806 21:11:49.947674 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 23067
I20260806 21:11:49.984280 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 22466
2026-08-06T21:11:49Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2372 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:11:50 dist-test-slave-lgr5 krb5kdc[23309](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:11:50 dist-test-slave-lgr5 krb5kdc[23309](info): set up 2 sockets
Aug 06 21:11:50 dist-test-slave-lgr5 krb5kdc[23309](info): commencing operation
krb5kdc: starting...
W20260806 21:11:51.137665 17096 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Aug 06 21:11:51 dist-test-slave-lgr5 krb5kdc[23309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050711, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:11:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:11:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.178.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
I20260806 21:11:51.267598 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:43615
--webserver_interface=127.16.178.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:35701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:43615
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:11:51.385342 23330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:51.385629 23330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:51.385692 23330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:51.389735 23330 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:11:51.389801 23330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:51.389827 23330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:51.389850 23330 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:11:51.389889 23330 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:11:51.395474 23330 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:35701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:43615
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:43615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.178.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:51.396660 23330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:51.397706 23330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:51.406283 23344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:51.406293 23342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:51.406411 23330 server_base.cc:1034] running on GCE node
W20260806 21:11:51.406283 23351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:51.407306 23330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:51.408651 23330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:51.409880 23330 hybrid_clock.cc:648] HybridClock initialized: now 1786050711409867 us; error 42 us; skew 500 ppm
Aug 06 21:11:51 dist-test-slave-lgr5 krb5kdc[23309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050711, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:51.412907 23330 init.cc:377] Logged in from keytab as kudu/127.16.178.62@KRBTEST.COM (short username kudu)
I20260806 21:11:51.413690 23330 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:51.413762 23330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:51.413969 23330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:51.416191 23330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "53b68f1d76dd4d2fac66e068bbfb1672"
format_stamp: "Formatted at 2026-08-06 21:11:51 on dist-test-slave-lgr5"
server_key: "b22357219b41f56e5d9f9a2b3077588c"
server_key_iv: "cc3a247000d625a3ee4385237469764f"
server_key_version: "encryptionkey@0"
I20260806 21:11:51.416787 23330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "53b68f1d76dd4d2fac66e068bbfb1672"
format_stamp: "Formatted at 2026-08-06 21:11:51 on dist-test-slave-lgr5"
server_key: "b22357219b41f56e5d9f9a2b3077588c"
server_key_iv: "cc3a247000d625a3ee4385237469764f"
server_key_version: "encryptionkey@0"
I20260806 21:11:51.420619 23330 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:11:51.423063 23357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:51.424373 23330 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 21:11:51.424491 23330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "53b68f1d76dd4d2fac66e068bbfb1672"
format_stamp: "Formatted at 2026-08-06 21:11:51 on dist-test-slave-lgr5"
server_key: "b22357219b41f56e5d9f9a2b3077588c"
server_key_iv: "cc3a247000d625a3ee4385237469764f"
server_key_version: "encryptionkey@0"
I20260806 21:11:51.424595 23330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:51.446601 23330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:51.449810 23330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:51.450024 23330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:51.457340 23330 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.62:43615
I20260806 21:11:51.457615 23446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.62:43615 every 8 connection(s)
I20260806 21:11:51.458112 23330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:11:51.461340 23447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:51.465731 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 23330
I20260806 21:11:51.465893 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:11:51.466193 17096 external_mini_cluster.cc:1428] Setting key 98097d0bb16bdf4477b5b0011a5d72a6
I20260806 21:11:51.468492 23447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53b68f1d76dd4d2fac66e068bbfb1672: Bootstrap starting.
I20260806 21:11:51.471127 23447 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53b68f1d76dd4d2fac66e068bbfb1672: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:51.471933 23447 log.cc:824] T 00000000000000000000000000000000 P 53b68f1d76dd4d2fac66e068bbfb1672: Log is configured to *not* fsync() on all Append() calls
I20260806 21:11:51.474177 23447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53b68f1d76dd4d2fac66e068bbfb1672: No bootstrap required, opened a new log
Aug 06 21:11:51 dist-test-slave-lgr5 krb5kdc[23309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050711, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:11:51.478134 23447 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53b68f1d76dd4d2fac66e068bbfb1672 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53b68f1d76dd4d2fac66e068bbfb1672" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 43615 } }
I20260806 21:11:51.478436 23447 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53b68f1d76dd4d2fac66e068bbfb1672 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:51.478499 23447 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53b68f1d76dd4d2fac66e068bbfb1672 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53b68f1d76dd4d2fac66e068bbfb1672, State: Initialized, Role: FOLLOWER
I20260806 21:11:51.479007 23447 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53b68f1d76dd4d2fac66e068bbfb1672 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53b68f1d76dd4d2fac66e068bbfb1672" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 43615 } }
I20260806 21:11:51.479172 23447 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53b68f1d76dd4d2fac66e068bbfb1672 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:11:51.479238 23447 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53b68f1d76dd4d2fac66e068bbfb1672 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:11:51.479336 23447 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53b68f1d76dd4d2fac66e068bbfb1672 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:51.480530 23447 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53b68f1d76dd4d2fac66e068bbfb1672 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53b68f1d76dd4d2fac66e068bbfb1672" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 43615 } }
I20260806 21:11:51.480929 23447 leader_election.cc:302] T 00000000000000000000000000000000 P 53b68f1d76dd4d2fac66e068bbfb1672 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53b68f1d76dd4d2fac66e068bbfb1672; no voters:
I20260806 21:11:51.481220 23452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:51.467803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38424 (local address 127.16.178.62:43615)
0806 21:11:51.468274 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:51.468291 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:51.468330 (+ 39us) server_negotiation.cc:407] Connection header received
0806 21:11:51.469039 (+ 709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:51.469058 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:51.469421 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:51.469704 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:51.470610 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:51.471679 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:51.472785 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:51.473253 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:51.473388 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:51.476199 (+ 2811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:51.476205 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:51.476219 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:51.476247 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:51.477837 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:51.478332 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:51.478337 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:51.478341 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:51.478464 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:51.478818 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:51.478822 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:51.478823 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:51.479385 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:51.479602 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:51.479801 (+ 199us) server_negotiation.cc:299] Negotiation successful
0806 21:11:51.480136 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":129,"threads_started":1}
I20260806 21:11:51.481339 23447 leader_election.cc:288] T 00000000000000000000000000000000 P 53b68f1d76dd4d2fac66e068bbfb1672 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:11:51.481479 23454 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53b68f1d76dd4d2fac66e068bbfb1672 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:11:51.481801 23454 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53b68f1d76dd4d2fac66e068bbfb1672 [term 1 LEADER]: Becoming Leader. State: Replica: 53b68f1d76dd4d2fac66e068bbfb1672, State: Running, Role: LEADER
I20260806 21:11:51.482151 23454 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53b68f1d76dd4d2fac66e068bbfb1672 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53b68f1d76dd4d2fac66e068bbfb1672" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 43615 } }
I20260806 21:11:51.482708 23447 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53b68f1d76dd4d2fac66e068bbfb1672 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:11:51.484167 23455 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53b68f1d76dd4d2fac66e068bbfb1672 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53b68f1d76dd4d2fac66e068bbfb1672" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53b68f1d76dd4d2fac66e068bbfb1672" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 43615 } } }
I20260806 21:11:51.484309 23455 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53b68f1d76dd4d2fac66e068bbfb1672 [sys.catalog]: This master's current role is: LEADER
I20260806 21:11:51.484720 23471 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:11:51.484748 23458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53b68f1d76dd4d2fac66e068bbfb1672 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53b68f1d76dd4d2fac66e068bbfb1672. Latest consensus state: current_term: 1 leader_uuid: "53b68f1d76dd4d2fac66e068bbfb1672" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53b68f1d76dd4d2fac66e068bbfb1672" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 43615 } } }
I20260806 21:11:51.484898 23458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53b68f1d76dd4d2fac66e068bbfb1672 [sys.catalog]: This master's current role is: LEADER
I20260806 21:11:51.487818 23471 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:11:51.493971 23471 catalog_manager.cc:1347] Generated new cluster ID: c48bbafbfb8e495e8f4eb56bbf07970d
I20260806 21:11:51.494074 23471 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:11:51.539479 23471 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:11:51.540738 23471 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:11:51.545945 23471 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53b68f1d76dd4d2fac66e068bbfb1672: Generated new TSK 0
I20260806 21:11:51.546645 23471 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.178.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab.
Entry for principal kudu/127.16.178.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab.
I20260806 21:11:51.583240 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.1:0
--local_ip_for_outbound_sockets=127.16.178.1
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:43615
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab
--principal=kudu/127.16.178.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:35701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:11:51.705325 23492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:51.705657 23492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:51.705708 23492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:51.709949 23492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:51.710053 23492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:51.710134 23492 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.1
I20260806 21:11:51.715766 23492 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:35701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab
--principal=kudu/127.16.178.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:43615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.1
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:51.716944 23492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:51.717983 23492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 21:11:51.727814 23492 server_base.cc:1034] running on GCE node
W20260806 21:11:51.727717 23505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:51.727717 23503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:51.727718 23502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:51.728571 23492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:51.729295 23492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:51.730495 23492 hybrid_clock.cc:648] HybridClock initialized: now 1786050711730468 us; error 35 us; skew 500 ppm
Aug 06 21:11:51 dist-test-slave-lgr5 krb5kdc[23309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050711, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:51.733474 23492 init.cc:377] Logged in from keytab as kudu/127.16.178.1@KRBTEST.COM (short username kudu)
I20260806 21:11:51.734300 23492 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:51.734383 23492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:51.734614 23492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:51.736877 23492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-0/data/instance:
uuid: "8ebb7ace4bb64664970a0abf7ecdc575"
format_stamp: "Formatted at 2026-08-06 21:11:51 on dist-test-slave-lgr5"
server_key: "47df7291dcd052f9a4cd51aaf44b56f1"
server_key_iv: "42e1299ca8a2a6443ebda4dc7db7a05d"
server_key_version: "encryptionkey@0"
I20260806 21:11:51.737458 23492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance:
uuid: "8ebb7ace4bb64664970a0abf7ecdc575"
format_stamp: "Formatted at 2026-08-06 21:11:51 on dist-test-slave-lgr5"
server_key: "47df7291dcd052f9a4cd51aaf44b56f1"
server_key_iv: "42e1299ca8a2a6443ebda4dc7db7a05d"
server_key_version: "encryptionkey@0"
I20260806 21:11:51.741349 23492 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:11:51.743867 23516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:51.745196 23492 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 21:11:51.745332 23492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-0/wal
uuid: "8ebb7ace4bb64664970a0abf7ecdc575"
format_stamp: "Formatted at 2026-08-06 21:11:51 on dist-test-slave-lgr5"
server_key: "47df7291dcd052f9a4cd51aaf44b56f1"
server_key_iv: "42e1299ca8a2a6443ebda4dc7db7a05d"
server_key_version: "encryptionkey@0"
I20260806 21:11:51.745448 23492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:51.765168 23492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:51.768608 23492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:51.768818 23492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:51.769470 23492 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:11:51.770370 23492 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:11:51.770419 23492 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:51.770478 23492 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:11:51.770503 23492 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:51.780206 23492 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.1:36115
I20260806 21:11:51.780233 23692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.1:36115 every 8 connection(s)
I20260806 21:11:51.780998 23492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
I20260806 21:11:51.781342 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 23492
I20260806 21:11:51.781497 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance
I20260806 21:11:51.781818 17096 external_mini_cluster.cc:1428] Setting key 6df558bbf6fa78d38ee77b80de617cdb
Aug 06 21:11:51 dist-test-slave-lgr5 krb5kdc[23309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050711, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.178.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.2@KRBTEST.COM" created.
I20260806 21:11:51.795332 23452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:51.782774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:37511 (local address 127.16.178.62:43615)
0806 21:11:51.782916 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:51.782920 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:51.784387 (+ 1467us) server_negotiation.cc:407] Connection header received
0806 21:11:51.785330 (+ 943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:51.785333 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:51.785390 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:51.785477 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:51.786756 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:51.787504 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:51.788776 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:51.789141 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:51.789261 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:51.791866 (+ 2605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:51.791870 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:51.791872 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:51.791898 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:51.793300 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:51.793763 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:51.793765 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:51.793766 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:51.793838 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:51.794114 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:51.794116 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:51.794117 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:51.794288 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:51.794377 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:51.794958 (+ 581us) server_negotiation.cc:299] Negotiation successful
0806 21:11:51.795135 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 21:11:51.796236 23698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:51.783267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:43615 (local address 127.16.178.1:37511)
0806 21:11:51.784243 (+ 976us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:51.784289 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:51.785124 (+ 835us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:51.785720 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:51.785727 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:51.786107 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:51.786603 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:51.786617 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:51.787622 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:51.787624 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:51.788644 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:51.788655 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:51.789327 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:51.789330 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:51.789483 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:51.790072 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:51.790089 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:51.791683 (+ 1594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:51.793411 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:51.793416 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:51.793431 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:51.793684 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:51.793907 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:51.793909 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:51.793911 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:51.794048 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:51.794381 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:51.794387 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:51.794680 (+ 293us) client_negotiation.cc:769] Sending connection context
0806 21:11:51.794853 (+ 173us) client_negotiation.cc:240] Negotiation successful
0806 21:11:51.795265 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":148,"threads_started":1}
I20260806 21:11:51.797724 23694 heartbeater.cc:347] Connected to a master server at 127.16.178.62:43615
I20260806 21:11:51.798039 23694 heartbeater.cc:464] Registering TS with master...
I20260806 21:11:51.798688 23694 heartbeater.cc:511] Master 127.16.178.62:43615 requested a full tablet report, sending...
I20260806 21:11:51.800544 23384 ts_manager.cc:194] Registered new tserver with Master: 8ebb7ace4bb64664970a0abf7ecdc575 (127.16.178.1:36115)
I20260806 21:11:51.801730 23384 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.1@KRBTEST.COM'} at 127.16.178.1:37511
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab.
Entry for principal kudu/127.16.178.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab.
I20260806 21:11:51.813257 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.2:0
--local_ip_for_outbound_sockets=127.16.178.2
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:43615
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab
--principal=kudu/127.16.178.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:35701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:11:51.928519 23702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:51.928795 23702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:51.928849 23702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:51.933713 23702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:51.933781 23702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:51.933846 23702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.2
I20260806 21:11:51.939230 23702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:35701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab
--principal=kudu/127.16.178.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:43615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.2
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:51.940376 23702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:51.941366 23702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:51.951073 23711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:51.951138 23702 server_base.cc:1034] running on GCE node
W20260806 21:11:51.951087 23712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:51.951064 23716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:51.951787 23702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:51.952564 23702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:51.953775 23702 hybrid_clock.cc:648] HybridClock initialized: now 1786050711953752 us; error 38 us; skew 500 ppm
Aug 06 21:11:51 dist-test-slave-lgr5 krb5kdc[23309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050711, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:51.956979 23702 init.cc:377] Logged in from keytab as kudu/127.16.178.2@KRBTEST.COM (short username kudu)
I20260806 21:11:51.957815 23702 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:51.957890 23702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:51.958103 23702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:51.960546 23702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-1/data/instance:
uuid: "9a98a34856904e9b88437d4cfd8d47ff"
format_stamp: "Formatted at 2026-08-06 21:11:51 on dist-test-slave-lgr5"
server_key: "0c4aecedfd9bf2afaa99fd4de027bff3"
server_key_iv: "1595d2260aa9731ba3947ecc12c1b0d3"
server_key_version: "encryptionkey@0"
I20260806 21:11:51.961165 23702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance:
uuid: "9a98a34856904e9b88437d4cfd8d47ff"
format_stamp: "Formatted at 2026-08-06 21:11:51 on dist-test-slave-lgr5"
server_key: "0c4aecedfd9bf2afaa99fd4de027bff3"
server_key_iv: "1595d2260aa9731ba3947ecc12c1b0d3"
server_key_version: "encryptionkey@0"
I20260806 21:11:51.965360 23702 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260806 21:11:51.968029 23724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:51.969640 23702 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 21:11:51.969781 23702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-1/wal
uuid: "9a98a34856904e9b88437d4cfd8d47ff"
format_stamp: "Formatted at 2026-08-06 21:11:51 on dist-test-slave-lgr5"
server_key: "0c4aecedfd9bf2afaa99fd4de027bff3"
server_key_iv: "1595d2260aa9731ba3947ecc12c1b0d3"
server_key_version: "encryptionkey@0"
I20260806 21:11:51.969908 23702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:52.001778 23702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:52.005277 23702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:52.005450 23702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:52.006045 23702 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:11:52.006919 23702 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:11:52.006978 23702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:52.007021 23702 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:11:52.007041 23702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:52.015921 23702 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.2:35575
I20260806 21:11:52.015965 23921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.2:35575 every 8 connection(s)
I20260806 21:11:52.016573 23702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
I20260806 21:11:52.021677 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 23702
I20260806 21:11:52.021786 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance
I20260806 21:11:52.022058 17096 external_mini_cluster.cc:1428] Setting key 2660c6c7d7b1d88580b3d767ca0d95d9
Aug 06 21:11:52 dist-test-slave-lgr5 krb5kdc[23309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050711, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.2@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:11:52.032159 23452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:52.018487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:59171 (local address 127.16.178.62:43615)
0806 21:11:52.018627 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:52.018631 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:52.019810 (+ 1179us) server_negotiation.cc:407] Connection header received
0806 21:11:52.020854 (+ 1044us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:52.020861 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:52.020934 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:52.021036 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:52.022464 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:52.023091 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:52.024633 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:52.024981 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:52.025134 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:52.028365 (+ 3231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:52.028368 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:52.028371 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:52.028400 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:52.029912 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:52.030478 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:52.030481 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:52.030483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:52.030563 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:52.030927 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:52.030930 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:52.030931 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:52.031125 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:52.031212 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:52.031796 (+ 584us) server_negotiation.cc:299] Negotiation successful
0806 21:11:52.031949 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
WARNING: no policy specified for kudu/127.16.178.3@KRBTEST.COM; defaulting to no policy
I20260806 21:11:52.033037 23925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:52.018858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:43615 (local address 127.16.178.2:59171)
0806 21:11:52.019655 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:52.019697 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:52.020567 (+ 870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:52.021251 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:52.021259 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:52.021672 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:52.022285 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:52.022304 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:52.023256 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:52.023259 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:52.024474 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:52.024489 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:52.025233 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:52.025237 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:52.025427 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:52.026182 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:52.026208 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:52.028157 (+ 1949us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:52.030045 (+ 1888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:52.030053 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:52.030073 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:52.030363 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:52.030650 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:52.030653 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:52.030655 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:52.030809 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:52.031229 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:52.031236 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:52.031526 (+ 290us) client_negotiation.cc:769] Sending connection context
0806 21:11:52.031705 (+ 179us) client_negotiation.cc:240] Negotiation successful
0806 21:11:52.032118 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":111,"threads_started":1}
I20260806 21:11:52.034163 23922 heartbeater.cc:347] Connected to a master server at 127.16.178.62:43615
I20260806 21:11:52.034415 23922 heartbeater.cc:464] Registering TS with master...
I20260806 21:11:52.034879 23922 heartbeater.cc:511] Master 127.16.178.62:43615 requested a full tablet report, sending...
I20260806 21:11:52.035921 23384 ts_manager.cc:194] Registered new tserver with Master: 9a98a34856904e9b88437d4cfd8d47ff (127.16.178.2:35575)
I20260806 21:11:52.036536 23384 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.2@KRBTEST.COM'} at 127.16.178.2:59171
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab.
Entry for principal kudu/127.16.178.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab.
W20260806 21:11:52.674113 17096 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.16.178.3: real 0.650s user 0.003s sys 0.002s
I20260806 21:11:52.674753 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.3:0
--local_ip_for_outbound_sockets=127.16.178.3
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:43615
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab
--principal=kudu/127.16.178.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:35701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:11:52.790867 23928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:52.791117 23928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:52.791180 23928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:52.795145 23928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:52.795213 23928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:52.795296 23928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.3
I20260806 21:11:52.800544 23928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:35701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab
--principal=kudu/127.16.178.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:43615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.3
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:52.801596 23928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:52.802646 23928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 21:11:52.804811 23694 heartbeater.cc:503] Master 127.16.178.62:43615 was elected leader, sending a full tablet report...
W20260806 21:11:52.811889 23937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:52.811903 23941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:52.811905 23935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:52.812207 23928 server_base.cc:1034] running on GCE node
I20260806 21:11:52.812698 23928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:52.813271 23928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:52.814465 23928 hybrid_clock.cc:648] HybridClock initialized: now 1786050712814451 us; error 38 us; skew 500 ppm
Aug 06 21:11:52 dist-test-slave-lgr5 krb5kdc[23309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050712, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:52.817160 23928 init.cc:377] Logged in from keytab as kudu/127.16.178.3@KRBTEST.COM (short username kudu)
I20260806 21:11:52.817873 23928 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:52.817945 23928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:52.818145 23928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:52.820245 23928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-2/data/instance:
uuid: "d1ae15829beb4dc88a019dfad58ccd5f"
format_stamp: "Formatted at 2026-08-06 21:11:52 on dist-test-slave-lgr5"
server_key: "68349282b529cc07f0d9878fd4aa4745"
server_key_iv: "ae2186a0d80ff2fef61d8211870e5971"
server_key_version: "encryptionkey@0"
I20260806 21:11:52.820802 23928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance:
uuid: "d1ae15829beb4dc88a019dfad58ccd5f"
format_stamp: "Formatted at 2026-08-06 21:11:52 on dist-test-slave-lgr5"
server_key: "68349282b529cc07f0d9878fd4aa4745"
server_key_iv: "ae2186a0d80ff2fef61d8211870e5971"
server_key_version: "encryptionkey@0"
I20260806 21:11:52.824383 23928 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 21:11:52.826694 23954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:52.828003 23928 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 21:11:52.828154 23928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-2/wal
uuid: "d1ae15829beb4dc88a019dfad58ccd5f"
format_stamp: "Formatted at 2026-08-06 21:11:52 on dist-test-slave-lgr5"
server_key: "68349282b529cc07f0d9878fd4aa4745"
server_key_iv: "ae2186a0d80ff2fef61d8211870e5971"
server_key_version: "encryptionkey@0"
I20260806 21:11:52.828253 23928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:52.841806 23928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:52.845661 23928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:52.845860 23928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:52.846531 23928 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:11:52.847486 23928 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:11:52.847546 23928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:52.847601 23928 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:11:52.847630 23928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:52.857707 23928 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.3:35259
I20260806 21:11:52.857736 24135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.3:35259 every 8 connection(s)
I20260806 21:11:52.858453 23928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
I20260806 21:11:52.862691 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 23928
I20260806 21:11:52.862847 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance
I20260806 21:11:52.863129 17096 external_mini_cluster.cc:1428] Setting key 421eb8a89f03e62ddaf3ada5fe806d6f
Aug 06 21:11:52 dist-test-slave-lgr5 krb5kdc[23309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050712, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.3@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:11:52.873001 24140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:52.860443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:55597 (local address 127.16.178.62:43615)
0806 21:11:52.860713 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:52.860717 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:52.861774 (+ 1057us) server_negotiation.cc:407] Connection header received
0806 21:11:52.862728 (+ 954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:52.862731 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:52.862790 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:52.862899 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:52.864454 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:52.865056 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:52.866353 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:52.866591 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:52.866685 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:52.869350 (+ 2665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:52.869354 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:52.869356 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:52.869382 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:52.870847 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:52.871383 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:52.871386 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:52.871393 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:52.871478 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:52.871822 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:52.871825 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:52.871827 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:52.872027 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:52.872137 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:52.872674 (+ 537us) server_negotiation.cc:299] Negotiation successful
0806 21:11:52.872816 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":63,"threads_started":1}
I20260806 21:11:52.873945 24142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:52.860915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:43615 (local address 127.16.178.3:55597)
0806 21:11:52.861610 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:52.861655 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:52.862470 (+ 815us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:52.863092 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:52.863100 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:52.863589 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:52.864286 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:52.864306 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:52.865193 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:52.865197 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:52.866194 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:52.866205 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:52.866751 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:52.866754 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:52.866911 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:52.867507 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:52.867524 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:52.869130 (+ 1606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:52.870984 (+ 1854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:52.870991 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:52.871009 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:52.871281 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:52.871567 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:52.871570 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:52.871572 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:52.871723 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:52.872153 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:52.872161 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:52.872432 (+ 271us) client_negotiation.cc:769] Sending connection context
0806 21:11:52.872614 (+ 182us) client_negotiation.cc:240] Negotiation successful
0806 21:11:52.872986 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":127,"threads_started":1}
I20260806 21:11:52.875152 24138 heartbeater.cc:347] Connected to a master server at 127.16.178.62:43615
I20260806 21:11:52.875398 24138 heartbeater.cc:464] Registering TS with master...
I20260806 21:11:52.875928 24138 heartbeater.cc:511] Master 127.16.178.62:43615 requested a full tablet report, sending...
I20260806 21:11:52.877123 23384 ts_manager.cc:194] Registered new tserver with Master: d1ae15829beb4dc88a019dfad58ccd5f (127.16.178.3:35259)
I20260806 21:11:52.877763 23384 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.3@KRBTEST.COM'} at 127.16.178.3:55597
I20260806 21:11:52.889448 17096 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 21:11:52.898696 24140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:52.891408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38436 (local address 127.16.178.62:43615)
0806 21:11:52.891566 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:52.891571 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:52.891702 (+ 131us) server_negotiation.cc:407] Connection header received
0806 21:11:52.891800 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:52.891802 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:52.891848 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:52.891915 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:52.892634 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:52.893158 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:52.893957 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:52.894184 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:52.894263 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:52.895621 (+ 1358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:52.895625 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:52.895627 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:52.895651 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:52.897014 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:52.897461 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:52.897463 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:52.897464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:52.897539 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:52.897728 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:52.897730 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:52.897731 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:52.897950 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:52.898048 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:52.898403 (+ 355us) server_negotiation.cc:299] Negotiation successful
0806 21:11:52.898530 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 21:11:52.901782 23384 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38436:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:11:52.904834 23384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:11:52.920905 24161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:52.915389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50030 (local address 127.16.178.1:36115)
0806 21:11:52.915795 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:52.915802 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:52.916059 (+ 257us) server_negotiation.cc:407] Connection header received
0806 21:11:52.916327 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:52.916333 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:52.916574 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:52.916698 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:11:52.917078 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:52.917960 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:52.919284 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:52.920177 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:52.920293 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:52.920452 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:52.920649 (+ 197us) server_negotiation.cc:299] Negotiation successful
0806 21:11:52.920748 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":113,"threads_started":1}
I20260806 21:11:52.920897 24156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:52.913954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.3:35259 (local address 127.0.0.1:58454)
0806 21:11:52.915082 (+ 1128us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:52.915140 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:52.915303 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:52.916197 (+ 894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:52.916204 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:52.916231 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:11:52.916378 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:52.916385 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:52.917901 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:52.917905 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:52.919327 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:52.919339 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:52.920528 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:52.920530 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:52.920603 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:52.920628 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 21:11:52.920698 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 21:11:52.920765 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":891,"mutex_wait_us":21,"thread_start_us":310,"threads_started":1}
I20260806 21:11:52.921016 24160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:52.914131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.2:35575 (local address 127.0.0.1:44010)
0806 21:11:52.915544 (+ 1413us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:52.915569 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:52.915665 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:52.916658 (+ 993us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:52.916680 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:52.916705 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:11:52.916909 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:52.916922 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:52.918388 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:52.918392 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:52.919560 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:52.919569 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:52.920603 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:52.920607 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:52.920690 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:52.920711 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 21:11:52.920774 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 21:11:52.920832 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1315,"mutex_wait_us":51,"thread_start_us":105,"threads_started":1}
I20260806 21:11:52.920907 24155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:52.915074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:36115 (local address 127.0.0.1:50030)
0806 21:11:52.915960 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:52.915973 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:52.916161 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:52.916875 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:52.916878 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:52.916895 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:11:52.916960 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:52.916966 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:52.918078 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:52.918082 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:52.919174 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:52.919184 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:52.920326 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:52.920330 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:52.920438 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:52.920476 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 21:11:52.920605 (+ 129us) client_negotiation.cc:240] Negotiation successful
0806 21:11:52.920741 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":824,"mutex_wait_us":1,"thread_start_us":165,"threads_started":1}
I20260806 21:11:52.921471 24157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:52.914270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58454 (local address 127.16.178.3:35259)
0806 21:11:52.915071 (+ 801us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:52.915089 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:52.915469 (+ 380us) server_negotiation.cc:407] Connection header received
0806 21:11:52.915569 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:52.915575 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:52.915804 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:52.915964 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:11:52.916531 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:52.917745 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:52.919451 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:52.920252 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:52.920347 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:52.920456 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:52.921243 (+ 787us) server_negotiation.cc:299] Negotiation successful
0806 21:11:52.921347 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":666,"thread_start_us":174,"threads_started":1}
I20260806 21:11:52.921711 24159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:52.914323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44010 (local address 127.16.178.2:35575)
0806 21:11:52.915992 (+ 1669us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:52.915998 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:52.916034 (+ 36us) server_negotiation.cc:407] Connection header received
0806 21:11:52.916224 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:52.916229 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:52.916464 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:52.916722 (+ 258us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:11:52.917078 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:52.918267 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:52.919673 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:52.920291 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:52.920391 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:52.920537 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:52.921442 (+ 905us) server_negotiation.cc:299] Negotiation successful
0806 21:11:52.921557 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1527,"thread_start_us":273,"threads_started":1}
I20260806 21:11:52.924108 24027 tablet_service.cc:1467] Processing CreateTablet for tablet ec83505ce9964745869ccd0d523f8971 (DEFAULT_TABLE table=test-table [id=7e40cc744b784186808529bb58dbbe37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:11:52.924204 23588 tablet_service.cc:1467] Processing CreateTablet for tablet ec83505ce9964745869ccd0d523f8971 (DEFAULT_TABLE table=test-table [id=7e40cc744b784186808529bb58dbbe37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:11:52.924247 23811 tablet_service.cc:1467] Processing CreateTablet for tablet ec83505ce9964745869ccd0d523f8971 (DEFAULT_TABLE table=test-table [id=7e40cc744b784186808529bb58dbbe37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:11:52.925223 24027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec83505ce9964745869ccd0d523f8971. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:52.925261 23588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec83505ce9964745869ccd0d523f8971. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:52.925274 23811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec83505ce9964745869ccd0d523f8971. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:52.932449 24163 tablet_bootstrap.cc:492] T ec83505ce9964745869ccd0d523f8971 P d1ae15829beb4dc88a019dfad58ccd5f: Bootstrap starting.
I20260806 21:11:52.933542 24164 tablet_bootstrap.cc:492] T ec83505ce9964745869ccd0d523f8971 P 8ebb7ace4bb64664970a0abf7ecdc575: Bootstrap starting.
I20260806 21:11:52.934077 24165 tablet_bootstrap.cc:492] T ec83505ce9964745869ccd0d523f8971 P 9a98a34856904e9b88437d4cfd8d47ff: Bootstrap starting.
I20260806 21:11:52.934825 24163 tablet_bootstrap.cc:654] T ec83505ce9964745869ccd0d523f8971 P d1ae15829beb4dc88a019dfad58ccd5f: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:52.935686 24164 tablet_bootstrap.cc:654] T ec83505ce9964745869ccd0d523f8971 P 8ebb7ace4bb64664970a0abf7ecdc575: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:52.935910 24163 log.cc:824] T ec83505ce9964745869ccd0d523f8971 P d1ae15829beb4dc88a019dfad58ccd5f: Log is configured to *not* fsync() on all Append() calls
I20260806 21:11:52.936427 24165 tablet_bootstrap.cc:654] T ec83505ce9964745869ccd0d523f8971 P 9a98a34856904e9b88437d4cfd8d47ff: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:52.936499 24164 log.cc:824] T ec83505ce9964745869ccd0d523f8971 P 8ebb7ace4bb64664970a0abf7ecdc575: Log is configured to *not* fsync() on all Append() calls
I20260806 21:11:52.937269 24165 log.cc:824] T ec83505ce9964745869ccd0d523f8971 P 9a98a34856904e9b88437d4cfd8d47ff: Log is configured to *not* fsync() on all Append() calls
I20260806 21:11:52.938194 24163 tablet_bootstrap.cc:492] T ec83505ce9964745869ccd0d523f8971 P d1ae15829beb4dc88a019dfad58ccd5f: No bootstrap required, opened a new log
I20260806 21:11:52.938402 24163 ts_tablet_manager.cc:1397] T ec83505ce9964745869ccd0d523f8971 P d1ae15829beb4dc88a019dfad58ccd5f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 21:11:52.938454 24164 tablet_bootstrap.cc:492] T ec83505ce9964745869ccd0d523f8971 P 8ebb7ace4bb64664970a0abf7ecdc575: No bootstrap required, opened a new log
I20260806 21:11:52.938620 24164 ts_tablet_manager.cc:1397] T ec83505ce9964745869ccd0d523f8971 P 8ebb7ace4bb64664970a0abf7ecdc575: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 21:11:52.938987 24165 tablet_bootstrap.cc:492] T ec83505ce9964745869ccd0d523f8971 P 9a98a34856904e9b88437d4cfd8d47ff: No bootstrap required, opened a new log
I20260806 21:11:52.939191 24165 ts_tablet_manager.cc:1397] T ec83505ce9964745869ccd0d523f8971 P 9a98a34856904e9b88437d4cfd8d47ff: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 21:11:52.942431 24163 raft_consensus.cc:348] T ec83505ce9964745869ccd0d523f8971 P d1ae15829beb4dc88a019dfad58ccd5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1ae15829beb4dc88a019dfad58ccd5f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 35259 } } peers { permanent_uuid: "8ebb7ace4bb64664970a0abf7ecdc575" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36115 } } peers { permanent_uuid: "9a98a34856904e9b88437d4cfd8d47ff" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 35575 } }
I20260806 21:11:52.942761 24163 raft_consensus.cc:374] T ec83505ce9964745869ccd0d523f8971 P d1ae15829beb4dc88a019dfad58ccd5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:52.942610 24165 raft_consensus.cc:348] T ec83505ce9964745869ccd0d523f8971 P 9a98a34856904e9b88437d4cfd8d47ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1ae15829beb4dc88a019dfad58ccd5f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 35259 } } peers { permanent_uuid: "8ebb7ace4bb64664970a0abf7ecdc575" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36115 } } peers { permanent_uuid: "9a98a34856904e9b88437d4cfd8d47ff" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 35575 } }
I20260806 21:11:52.942824 24163 raft_consensus.cc:729] T ec83505ce9964745869ccd0d523f8971 P d1ae15829beb4dc88a019dfad58ccd5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1ae15829beb4dc88a019dfad58ccd5f, State: Initialized, Role: FOLLOWER
I20260806 21:11:52.942843 24165 raft_consensus.cc:374] T ec83505ce9964745869ccd0d523f8971 P 9a98a34856904e9b88437d4cfd8d47ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:52.942898 24165 raft_consensus.cc:729] T ec83505ce9964745869ccd0d523f8971 P 9a98a34856904e9b88437d4cfd8d47ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a98a34856904e9b88437d4cfd8d47ff, State: Initialized, Role: FOLLOWER
I20260806 21:11:52.942782 24164 raft_consensus.cc:348] T ec83505ce9964745869ccd0d523f8971 P 8ebb7ace4bb64664970a0abf7ecdc575 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1ae15829beb4dc88a019dfad58ccd5f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 35259 } } peers { permanent_uuid: "8ebb7ace4bb64664970a0abf7ecdc575" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36115 } } peers { permanent_uuid: "9a98a34856904e9b88437d4cfd8d47ff" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 35575 } }
I20260806 21:11:52.943020 24164 raft_consensus.cc:374] T ec83505ce9964745869ccd0d523f8971 P 8ebb7ace4bb64664970a0abf7ecdc575 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:52.943078 24164 raft_consensus.cc:729] T ec83505ce9964745869ccd0d523f8971 P 8ebb7ace4bb64664970a0abf7ecdc575 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ebb7ace4bb64664970a0abf7ecdc575, State: Initialized, Role: FOLLOWER
I20260806 21:11:52.943331 24163 consensus_queue.cc:260] T ec83505ce9964745869ccd0d523f8971 P d1ae15829beb4dc88a019dfad58ccd5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1ae15829beb4dc88a019dfad58ccd5f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 35259 } } peers { permanent_uuid: "8ebb7ace4bb64664970a0abf7ecdc575" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36115 } } peers { permanent_uuid: "9a98a34856904e9b88437d4cfd8d47ff" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 35575 } }
I20260806 21:11:52.943377 24165 consensus_queue.cc:260] T ec83505ce9964745869ccd0d523f8971 P 9a98a34856904e9b88437d4cfd8d47ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1ae15829beb4dc88a019dfad58ccd5f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 35259 } } peers { permanent_uuid: "8ebb7ace4bb64664970a0abf7ecdc575" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36115 } } peers { permanent_uuid: "9a98a34856904e9b88437d4cfd8d47ff" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 35575 } }
I20260806 21:11:52.943442 24164 consensus_queue.cc:260] T ec83505ce9964745869ccd0d523f8971 P 8ebb7ace4bb64664970a0abf7ecdc575 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1ae15829beb4dc88a019dfad58ccd5f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 35259 } } peers { permanent_uuid: "8ebb7ace4bb64664970a0abf7ecdc575" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36115 } } peers { permanent_uuid: "9a98a34856904e9b88437d4cfd8d47ff" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 35575 } }
I20260806 21:11:52.944411 24165 ts_tablet_manager.cc:1428] T ec83505ce9964745869ccd0d523f8971 P 9a98a34856904e9b88437d4cfd8d47ff: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 21:11:52.944411 24163 ts_tablet_manager.cc:1428] T ec83505ce9964745869ccd0d523f8971 P d1ae15829beb4dc88a019dfad58ccd5f: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 21:11:52.944411 24164 ts_tablet_manager.cc:1428] T ec83505ce9964745869ccd0d523f8971 P 8ebb7ace4bb64664970a0abf7ecdc575: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 21:11:52.944787 23922 heartbeater.cc:503] Master 127.16.178.62:43615 was elected leader, sending a full tablet report...
I20260806 21:11:52.945794 24138 heartbeater.cc:503] Master 127.16.178.62:43615 was elected leader, sending a full tablet report...
I20260806 21:11:53.003070 24170 raft_consensus.cc:482] T ec83505ce9964745869ccd0d523f8971 P 9a98a34856904e9b88437d4cfd8d47ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:11:53.003316 24170 raft_consensus.cc:504] T ec83505ce9964745869ccd0d523f8971 P 9a98a34856904e9b88437d4cfd8d47ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1ae15829beb4dc88a019dfad58ccd5f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 35259 } } peers { permanent_uuid: "8ebb7ace4bb64664970a0abf7ecdc575" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36115 } } peers { permanent_uuid: "9a98a34856904e9b88437d4cfd8d47ff" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 35575 } }
I20260806 21:11:53.004539 24170 leader_election.cc:288] T ec83505ce9964745869ccd0d523f8971 P 9a98a34856904e9b88437d4cfd8d47ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1ae15829beb4dc88a019dfad58ccd5f (127.16.178.3:35259), 8ebb7ace4bb64664970a0abf7ecdc575 (127.16.178.1:36115)
I20260806 21:11:53.009284 24173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:53.004855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.3:35259 (local address 127.16.178.2:56901)
0806 21:11:53.005194 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:53.005208 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:53.005329 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:53.005662 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:53.005665 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:53.005686 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:11:53.005826 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:53.005832 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:53.006787 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:53.006790 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:53.008170 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:53.008176 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:53.008838 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:53.008842 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:53.008940 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:53.008962 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 21:11:53.009034 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 21:11:53.009107 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":24,"thread_start_us":154,"threads_started":1}
I20260806 21:11:53.009289 24157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:53.004963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:56901 (local address 127.16.178.3:35259)
0806 21:11:53.005104 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:53.005108 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:53.005424 (+ 316us) server_negotiation.cc:407] Connection header received
0806 21:11:53.005492 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:53.005495 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:53.005551 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:53.005630 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:11:53.005927 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:53.006681 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:53.008320 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:53.008722 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:53.008834 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:53.008889 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:53.009077 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 21:11:53.009161 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 21:11:53.009452 24174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:53.005322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:36115 (local address 127.16.178.2:46043)
0806 21:11:53.005605 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:53.005618 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:53.005708 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:53.005937 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:53.005940 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:53.005958 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:11:53.006042 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:53.006046 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:53.006939 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:53.006942 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:53.008164 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:53.008176 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:53.009058 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:53.009062 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:53.009169 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:53.009191 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 21:11:53.009261 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 21:11:53.009321 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"mutex_wait_us":23,"thread_start_us":99,"threads_started":1}
I20260806 21:11:53.009497 24161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:53.005450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:46043 (local address 127.16.178.1:36115)
0806 21:11:53.005573 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:53.005577 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:53.005695 (+ 118us) server_negotiation.cc:407] Connection header received
0806 21:11:53.005799 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:53.005802 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:53.005844 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:53.005913 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:11:53.006134 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:53.006850 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:53.008343 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:53.008935 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:53.009072 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:53.009126 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:53.009293 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 21:11:53.009372 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 21:11:53.010025 24057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec83505ce9964745869ccd0d523f8971" candidate_uuid: "9a98a34856904e9b88437d4cfd8d47ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1ae15829beb4dc88a019dfad58ccd5f" is_pre_election: true
I20260806 21:11:53.010085 23615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec83505ce9964745869ccd0d523f8971" candidate_uuid: "9a98a34856904e9b88437d4cfd8d47ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebb7ace4bb64664970a0abf7ecdc575" is_pre_election: true
I20260806 21:11:53.010340 24057 raft_consensus.cc:2454] T ec83505ce9964745869ccd0d523f8971 P d1ae15829beb4dc88a019dfad58ccd5f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a98a34856904e9b88437d4cfd8d47ff in term 0.
I20260806 21:11:53.010411 23615 raft_consensus.cc:2454] T ec83505ce9964745869ccd0d523f8971 P 8ebb7ace4bb64664970a0abf7ecdc575 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a98a34856904e9b88437d4cfd8d47ff in term 0.
I20260806 21:11:53.010869 23727 leader_election.cc:302] T ec83505ce9964745869ccd0d523f8971 P 9a98a34856904e9b88437d4cfd8d47ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a98a34856904e9b88437d4cfd8d47ff, d1ae15829beb4dc88a019dfad58ccd5f; no voters:
I20260806 21:11:53.011137 24170 raft_consensus.cc:2781] T ec83505ce9964745869ccd0d523f8971 P 9a98a34856904e9b88437d4cfd8d47ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:11:53.011238 24170 raft_consensus.cc:482] T ec83505ce9964745869ccd0d523f8971 P 9a98a34856904e9b88437d4cfd8d47ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:11:53.011307 24170 raft_consensus.cc:3037] T ec83505ce9964745869ccd0d523f8971 P 9a98a34856904e9b88437d4cfd8d47ff [term 0 FOLLOWER]: Advancing to term 1
W20260806 21:11:53.018144 23924 tablet.cc:2369] T ec83505ce9964745869ccd0d523f8971 P 9a98a34856904e9b88437d4cfd8d47ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 21:11:53.032792 23695 tablet.cc:2369] T ec83505ce9964745869ccd0d523f8971 P 8ebb7ace4bb64664970a0abf7ecdc575: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 21:11:53.071269 24170 raft_consensus.cc:504] T ec83505ce9964745869ccd0d523f8971 P 9a98a34856904e9b88437d4cfd8d47ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1ae15829beb4dc88a019dfad58ccd5f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 35259 } } peers { permanent_uuid: "8ebb7ace4bb64664970a0abf7ecdc575" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36115 } } peers { permanent_uuid: "9a98a34856904e9b88437d4cfd8d47ff" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 35575 } }
I20260806 21:11:53.071884 24170 leader_election.cc:288] T ec83505ce9964745869ccd0d523f8971 P 9a98a34856904e9b88437d4cfd8d47ff [CANDIDATE]: Term 1 election: Requested vote from peers d1ae15829beb4dc88a019dfad58ccd5f (127.16.178.3:35259), 8ebb7ace4bb64664970a0abf7ecdc575 (127.16.178.1:36115)
I20260806 21:11:53.072518 23615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec83505ce9964745869ccd0d523f8971" candidate_uuid: "9a98a34856904e9b88437d4cfd8d47ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ebb7ace4bb64664970a0abf7ecdc575"
I20260806 21:11:53.072562 24057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec83505ce9964745869ccd0d523f8971" candidate_uuid: "9a98a34856904e9b88437d4cfd8d47ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1ae15829beb4dc88a019dfad58ccd5f"
I20260806 21:11:53.072691 23615 raft_consensus.cc:3037] T ec83505ce9964745869ccd0d523f8971 P 8ebb7ace4bb64664970a0abf7ecdc575 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:53.072729 24057 raft_consensus.cc:3037] T ec83505ce9964745869ccd0d523f8971 P d1ae15829beb4dc88a019dfad58ccd5f [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:53.074158 24057 raft_consensus.cc:2454] T ec83505ce9964745869ccd0d523f8971 P d1ae15829beb4dc88a019dfad58ccd5f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a98a34856904e9b88437d4cfd8d47ff in term 1.
I20260806 21:11:53.074309 23615 raft_consensus.cc:2454] T ec83505ce9964745869ccd0d523f8971 P 8ebb7ace4bb64664970a0abf7ecdc575 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a98a34856904e9b88437d4cfd8d47ff in term 1.
I20260806 21:11:53.075646 23727 leader_election.cc:302] T ec83505ce9964745869ccd0d523f8971 P 9a98a34856904e9b88437d4cfd8d47ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a98a34856904e9b88437d4cfd8d47ff, d1ae15829beb4dc88a019dfad58ccd5f; no voters:
I20260806 21:11:53.075896 24170 raft_consensus.cc:2781] T ec83505ce9964745869ccd0d523f8971 P 9a98a34856904e9b88437d4cfd8d47ff [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:11:53.076471 24170 raft_consensus.cc:686] T ec83505ce9964745869ccd0d523f8971 P 9a98a34856904e9b88437d4cfd8d47ff [term 1 LEADER]: Becoming Leader. State: Replica: 9a98a34856904e9b88437d4cfd8d47ff, State: Running, Role: LEADER
I20260806 21:11:53.077123 24170 consensus_queue.cc:237] T ec83505ce9964745869ccd0d523f8971 P 9a98a34856904e9b88437d4cfd8d47ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1ae15829beb4dc88a019dfad58ccd5f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 35259 } } peers { permanent_uuid: "8ebb7ace4bb64664970a0abf7ecdc575" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36115 } } peers { permanent_uuid: "9a98a34856904e9b88437d4cfd8d47ff" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 35575 } }
I20260806 21:11:53.081231 23384 catalog_manager.cc:5495] T ec83505ce9964745869ccd0d523f8971 P 9a98a34856904e9b88437d4cfd8d47ff reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a98a34856904e9b88437d4cfd8d47ff (127.16.178.2). New cstate: current_term: 1 leader_uuid: "9a98a34856904e9b88437d4cfd8d47ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1ae15829beb4dc88a019dfad58ccd5f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 35259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ebb7ace4bb64664970a0abf7ecdc575" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a98a34856904e9b88437d4cfd8d47ff" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 35575 } health_report { overall_health: HEALTHY } } }
W20260806 21:11:53.109398 24139 tablet.cc:2369] T ec83505ce9964745869ccd0d523f8971 P d1ae15829beb4dc88a019dfad58ccd5f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 21:11:53.112313 24159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:53.108435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44014 (local address 127.16.178.2:35575)
0806 21:11:53.108565 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:53.108570 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:53.108584 (+ 14us) server_negotiation.cc:407] Connection header received
0806 21:11:53.108669 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:53.108671 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:53.108725 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:53.108847 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:11:53.109078 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:53.109761 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:53.110665 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:53.110898 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:53.111001 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:53.111355 (+ 354us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:11:53.111867 (+ 512us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:11:53.111979 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:53.112128 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 21:11:53.112197 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 21:11:53.118628 23615 raft_consensus.cc:1261] T ec83505ce9964745869ccd0d523f8971 P 8ebb7ace4bb64664970a0abf7ecdc575 [term 1 FOLLOWER]: Refusing update from remote peer 9a98a34856904e9b88437d4cfd8d47ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:11:53.118628 24057 raft_consensus.cc:1261] T ec83505ce9964745869ccd0d523f8971 P d1ae15829beb4dc88a019dfad58ccd5f [term 1 FOLLOWER]: Refusing update from remote peer 9a98a34856904e9b88437d4cfd8d47ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:11:53.119465 24176 consensus_queue.cc:1035] T ec83505ce9964745869ccd0d523f8971 P 9a98a34856904e9b88437d4cfd8d47ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ebb7ace4bb64664970a0abf7ecdc575" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:11:53.119658 24170 consensus_queue.cc:1035] T ec83505ce9964745869ccd0d523f8971 P 9a98a34856904e9b88437d4cfd8d47ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1ae15829beb4dc88a019dfad58ccd5f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 35259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:11:53.128058 24185 mvcc.cc:205] Tried to move back new op lower bound from 7315663720925171712 to 7315663720767512576. Current Snapshot: MvccSnapshot[applied={T|T < 7315663720925171712}]
I20260806 21:11:53.137928 23383 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:11:53.138145 23383 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:11:53.140772 23383 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 53b68f1d76dd4d2fac66e068bbfb1672: Sending DeleteTablet for 3 replicas of tablet ec83505ce9964745869ccd0d523f8971
I20260806 21:11:53.141561 23811 tablet_service.cc:1514] Processing DeleteTablet for tablet ec83505ce9964745869ccd0d523f8971 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:11:53 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:44010
I20260806 21:11:53.141721 23588 tablet_service.cc:1514] Processing DeleteTablet for tablet ec83505ce9964745869ccd0d523f8971 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:11:53 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:50030
I20260806 21:11:53.141906 24027 tablet_service.cc:1514] Processing DeleteTablet for tablet ec83505ce9964745869ccd0d523f8971 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:11:53 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:58454
I20260806 21:11:53.142163 24202 tablet_replica.cc:330] T ec83505ce9964745869ccd0d523f8971 P 8ebb7ace4bb64664970a0abf7ecdc575: stopping tablet replica
I20260806 21:11:53.142504 24202 raft_consensus.cc:2229] T ec83505ce9964745869ccd0d523f8971 P 8ebb7ace4bb64664970a0abf7ecdc575 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:11:53.142596 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 23492
I20260806 21:11:53.142815 24205 tablet_replica.cc:330] T ec83505ce9964745869ccd0d523f8971 P d1ae15829beb4dc88a019dfad58ccd5f: stopping tablet replica
I20260806 21:11:53.142855 24202 raft_consensus.cc:2258] T ec83505ce9964745869ccd0d523f8971 P 8ebb7ace4bb64664970a0abf7ecdc575 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:11:53.143013 24201 tablet_replica.cc:330] T ec83505ce9964745869ccd0d523f8971 P 9a98a34856904e9b88437d4cfd8d47ff: stopping tablet replica
I20260806 21:11:53.143323 24201 raft_consensus.cc:2229] T ec83505ce9964745869ccd0d523f8971 P 9a98a34856904e9b88437d4cfd8d47ff [term 1 LEADER]: Raft consensus shutting down.
I20260806 21:11:53.143664 24205 raft_consensus.cc:2229] T ec83505ce9964745869ccd0d523f8971 P d1ae15829beb4dc88a019dfad58ccd5f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:11:53.143978 24205 raft_consensus.cc:2258] T ec83505ce9964745869ccd0d523f8971 P d1ae15829beb4dc88a019dfad58ccd5f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:11:53.144563 24201 raft_consensus.cc:2258] T ec83505ce9964745869ccd0d523f8971 P 9a98a34856904e9b88437d4cfd8d47ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:11:53.149803 24205 ts_tablet_manager.cc:1905] T ec83505ce9964745869ccd0d523f8971 P d1ae15829beb4dc88a019dfad58ccd5f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:11:53.149957 24201 ts_tablet_manager.cc:1905] T ec83505ce9964745869ccd0d523f8971 P 9a98a34856904e9b88437d4cfd8d47ff: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 21:11:53.164149 23362 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.178.1:36115 (error 108)
I20260806 21:11:53.164820 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 23702
I20260806 21:11:53.169318 24160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:53.168828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:36115 (local address 127.0.0.1:50038)
0806 21:11:53.169089 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:53.169196 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.1:36115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157}
W20260806 21:11:53.169786 23362 catalog_manager.cc:4584] TS 8ebb7ace4bb64664970a0abf7ecdc575 (127.16.178.1:36115): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ec83505ce9964745869ccd0d523f8971: Network error: Client connection negotiation failed: client connection to 127.16.178.1:36115: connect: Connection refused (error 111)
I20260806 21:11:53.175098 24205 ts_tablet_manager.cc:1918] T ec83505ce9964745869ccd0d523f8971 P d1ae15829beb4dc88a019dfad58ccd5f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:11:53.175237 24205 log.cc:1196] T ec83505ce9964745869ccd0d523f8971 P d1ae15829beb4dc88a019dfad58ccd5f: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786050686797304-17096-0/minicluster-data/ts-2/wal/wals/ec83505ce9964745869ccd0d523f8971
I20260806 21:11:53.175554 24205 ts_tablet_manager.cc:1939] T ec83505ce9964745869ccd0d523f8971 P d1ae15829beb4dc88a019dfad58ccd5f: Deleting consensus metadata
I20260806 21:11:53.181250 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 23928
I20260806 21:11:53.193773 24160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:53.181398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.2:35575 (local address 127.0.0.1:44022)
0806 21:11:53.193542 (+ 12144us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:53.193649 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.2:35575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11984}
I20260806 21:11:53.200665 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 23330
2026-08-06T21:11:53Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (3215 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-06T21:11:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:11:53Z Disabled control of system clock
I20260806 21:11:53.239639 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:37227
--webserver_interface=127.16.178.62
--webserver_port=0
--builtin_ntp_servers=127.16.178.20:40695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:37227
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 21:11:53.363325 24211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:53.363641 24211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:53.363696 24211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:53.368247 24211 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:11:53.368336 24211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:53.368359 24211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:53.368381 24211 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:11:53.368402 24211 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:11:53.373838 24211 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:40695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:37227
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:37227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.178.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24211
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:53.375051 24211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:53.376609 24211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:53.385350 24223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:53.385375 24221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:53.385350 24219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:53.385561 24211 server_base.cc:1034] running on GCE node
I20260806 21:11:53.386245 24211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:53.387383 24211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:53.388552 24211 hybrid_clock.cc:648] HybridClock initialized: now 1786050713388535 us; error 35 us; skew 500 ppm
I20260806 21:11:53.390527 24211 webserver.cc:460] Webserver started at http://127.16.178.62:45263/ using document root <none> and password file <none>
I20260806 21:11:53.391098 24211 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:53.391168 24211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:53.391372 24211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:53.393474 24211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "9269fe44d4364694bdef0821ae8b6346"
format_stamp: "Formatted at 2026-08-06 21:11:53 on dist-test-slave-lgr5"
server_key: "7b3a9aacac54d052a39314bb43d9e936"
server_key_iv: "3a6afc359d3c149d9ae2d267753fd60b"
server_key_version: "encryptionkey@0"
I20260806 21:11:53.394009 24211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "9269fe44d4364694bdef0821ae8b6346"
format_stamp: "Formatted at 2026-08-06 21:11:53 on dist-test-slave-lgr5"
server_key: "7b3a9aacac54d052a39314bb43d9e936"
server_key_iv: "3a6afc359d3c149d9ae2d267753fd60b"
server_key_version: "encryptionkey@0"
I20260806 21:11:53.397956 24211 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:11:53.400398 24230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:53.401618 24211 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 21:11:53.401734 24211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "9269fe44d4364694bdef0821ae8b6346"
format_stamp: "Formatted at 2026-08-06 21:11:53 on dist-test-slave-lgr5"
server_key: "7b3a9aacac54d052a39314bb43d9e936"
server_key_iv: "3a6afc359d3c149d9ae2d267753fd60b"
server_key_version: "encryptionkey@0"
I20260806 21:11:53.401835 24211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:53.406530 24211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:53.407126 24211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:53.407289 24211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:53.414021 24211 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.62:37227
I20260806 21:11:53.414047 24314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.62:37227 every 8 connection(s)
I20260806 21:11:53.415058 24211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:11:53.416751 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 24211
I20260806 21:11:53.416862 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:11:53.417157 17096 external_mini_cluster.cc:1428] Setting key 5110b086867efa7889b93e9169f3c31c
I20260806 21:11:53.418915 24315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:53.424134 24318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:53.418605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55500 (local address 127.16.178.62:37227)
0806 21:11:53.419342 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:53.419363 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:53.419409 (+ 46us) server_negotiation.cc:407] Connection header received
0806 21:11:53.420404 (+ 995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:53.420455 (+ 51us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:53.420997 (+ 542us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:53.421469 (+ 472us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:53.422278 (+ 809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:53.422284 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:53.422300 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:11:53.422313 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:53.422396 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:11:53.422422 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:53.422586 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:53.422704 (+ 118us) server_negotiation.cc:299] Negotiation successful
0806 21:11:53.423018 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":116,"threads_started":1}
I20260806 21:11:53.426994 24315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9269fe44d4364694bdef0821ae8b6346: Bootstrap starting.
I20260806 21:11:53.429690 24315 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9269fe44d4364694bdef0821ae8b6346: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:53.430480 24315 log.cc:824] T 00000000000000000000000000000000 P 9269fe44d4364694bdef0821ae8b6346: Log is configured to *not* fsync() on all Append() calls
I20260806 21:11:53.433007 24315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9269fe44d4364694bdef0821ae8b6346: No bootstrap required, opened a new log
I20260806 21:11:53.436954 24315 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9269fe44d4364694bdef0821ae8b6346 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9269fe44d4364694bdef0821ae8b6346" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 37227 } }
I20260806 21:11:53.437222 24315 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9269fe44d4364694bdef0821ae8b6346 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:53.437266 24315 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9269fe44d4364694bdef0821ae8b6346 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9269fe44d4364694bdef0821ae8b6346, State: Initialized, Role: FOLLOWER
I20260806 21:11:53.437664 24315 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9269fe44d4364694bdef0821ae8b6346 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9269fe44d4364694bdef0821ae8b6346" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 37227 } }
I20260806 21:11:53.437801 24315 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9269fe44d4364694bdef0821ae8b6346 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:11:53.437847 24315 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9269fe44d4364694bdef0821ae8b6346 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:11:53.437942 24315 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9269fe44d4364694bdef0821ae8b6346 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:53.471395 24315 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9269fe44d4364694bdef0821ae8b6346 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9269fe44d4364694bdef0821ae8b6346" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 37227 } }
I20260806 21:11:53.471966 24315 leader_election.cc:302] T 00000000000000000000000000000000 P 9269fe44d4364694bdef0821ae8b6346 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9269fe44d4364694bdef0821ae8b6346; no voters:
I20260806 21:11:53.472429 24315 leader_election.cc:288] T 00000000000000000000000000000000 P 9269fe44d4364694bdef0821ae8b6346 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:11:53.472616 24320 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9269fe44d4364694bdef0821ae8b6346 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:11:53.472852 24320 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9269fe44d4364694bdef0821ae8b6346 [term 1 LEADER]: Becoming Leader. State: Replica: 9269fe44d4364694bdef0821ae8b6346, State: Running, Role: LEADER
I20260806 21:11:53.473241 24320 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9269fe44d4364694bdef0821ae8b6346 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9269fe44d4364694bdef0821ae8b6346" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 37227 } }
I20260806 21:11:53.473898 24315 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9269fe44d4364694bdef0821ae8b6346 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:11:53.475231 24322 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9269fe44d4364694bdef0821ae8b6346 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9269fe44d4364694bdef0821ae8b6346. Latest consensus state: current_term: 1 leader_uuid: "9269fe44d4364694bdef0821ae8b6346" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9269fe44d4364694bdef0821ae8b6346" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 37227 } } }
I20260806 21:11:53.475294 24321 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9269fe44d4364694bdef0821ae8b6346 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9269fe44d4364694bdef0821ae8b6346" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9269fe44d4364694bdef0821ae8b6346" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 37227 } } }
I20260806 21:11:53.475394 24322 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9269fe44d4364694bdef0821ae8b6346 [sys.catalog]: This master's current role is: LEADER
I20260806 21:11:53.475404 24321 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9269fe44d4364694bdef0821ae8b6346 [sys.catalog]: This master's current role is: LEADER
I20260806 21:11:53.475821 24336 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:11:53.479015 24336 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:11:53.485241 24336 catalog_manager.cc:1347] Generated new cluster ID: 1e2a39a320ea4032a3b7a90eb2a2d66c
I20260806 21:11:53.485332 24336 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:11:53.503437 24336 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:11:53.503863 24336 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:11:53.512629 24336 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9269fe44d4364694bdef0821ae8b6346: Generated new TSK 0
I20260806 21:11:53.513767 24336 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:11:53.529105 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.1:0
--local_ip_for_outbound_sockets=127.16.178.1
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:37227
--builtin_ntp_servers=127.16.178.20:40695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 21:11:53.650276 24347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:53.650576 24347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:53.650616 24347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:53.654722 24347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:53.654789 24347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:53.654848 24347 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.1
I20260806 21:11:53.659642 24347 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:40695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:37227
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.1
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:53.660821 24347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:53.662288 24347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:53.672061 24354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:53.672066 24358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:53.672056 24356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:53.672426 24347 server_base.cc:1034] running on GCE node
I20260806 21:11:53.672943 24347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:53.673671 24347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:53.674835 24347 hybrid_clock.cc:648] HybridClock initialized: now 1786050713674811 us; error 34 us; skew 500 ppm
I20260806 21:11:53.676888 24347 webserver.cc:460] Webserver started at http://127.16.178.1:40943/ using document root <none> and password file <none>
I20260806 21:11:53.677466 24347 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:53.677525 24347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:53.677701 24347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:53.679791 24347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-0/data/instance:
uuid: "94ccbe9c249c4b0eb7209cff25f7928c"
format_stamp: "Formatted at 2026-08-06 21:11:53 on dist-test-slave-lgr5"
server_key: "18f8f7f1aa7d0d5138ed82f38ec64c4e"
server_key_iv: "8ad9b57dbd550a10dab2c400d1f11053"
server_key_version: "encryptionkey@0"
I20260806 21:11:53.680497 24347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance:
uuid: "94ccbe9c249c4b0eb7209cff25f7928c"
format_stamp: "Formatted at 2026-08-06 21:11:53 on dist-test-slave-lgr5"
server_key: "18f8f7f1aa7d0d5138ed82f38ec64c4e"
server_key_iv: "8ad9b57dbd550a10dab2c400d1f11053"
server_key_version: "encryptionkey@0"
I20260806 21:11:53.684065 24347 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:11:53.686592 24366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:53.687764 24347 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 21:11:53.687863 24347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-0/wal
uuid: "94ccbe9c249c4b0eb7209cff25f7928c"
format_stamp: "Formatted at 2026-08-06 21:11:53 on dist-test-slave-lgr5"
server_key: "18f8f7f1aa7d0d5138ed82f38ec64c4e"
server_key_iv: "8ad9b57dbd550a10dab2c400d1f11053"
server_key_version: "encryptionkey@0"
I20260806 21:11:53.687952 24347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:53.692446 24347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:53.693007 24347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:53.693142 24347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:53.693641 24347 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:11:53.694510 24347 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:11:53.694555 24347 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:53.694592 24347 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:11:53.694612 24347 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:53.703075 24347 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.1:40607
I20260806 21:11:53.703099 24534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.1:40607 every 8 connection(s)
I20260806 21:11:53.703954 24347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
I20260806 21:11:53.706725 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 24347
I20260806 21:11:53.706851 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance
I20260806 21:11:53.707175 17096 external_mini_cluster.cc:1428] Setting key 32d2dddb8057277b12c7a8d9a4ec6664
I20260806 21:11:53.709405 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.2:0
--local_ip_for_outbound_sockets=127.16.178.2
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:37227
--builtin_ntp_servers=127.16.178.20:40695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 21:11:53.710692 24318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:53.705863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:49473 (local address 127.16.178.62:37227)
0806 21:11:53.705998 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:53.706001 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:53.707258 (+ 1257us) server_negotiation.cc:407] Connection header received
0806 21:11:53.708948 (+ 1690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:53.708951 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:53.708983 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:53.709047 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:53.710286 (+ 1239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:53.710289 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:53.710291 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:11:53.710303 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:53.710325 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:11:53.710333 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:53.710417 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:53.710555 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 21:11:53.710590 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 21:11:53.711707 24540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:53.706298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37227 (local address 127.16.178.1:49473)
0806 21:11:53.707094 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:53.707148 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:53.708730 (+ 1582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:53.709235 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:53.709245 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:53.709361 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:53.709962 (+ 601us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 21:11:53.709964 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:53.710032 (+ 68us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 21:11:53.710123 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:53.710419 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:53.710427 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:53.710433 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 21:11:53.710601 (+ 168us) client_negotiation.cc:240] Negotiation successful
0806 21:11:53.710842 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":117,"threads_started":1}
I20260806 21:11:53.712889 24537 heartbeater.cc:347] Connected to a master server at 127.16.178.62:37227
I20260806 21:11:53.713131 24537 heartbeater.cc:464] Registering TS with master...
I20260806 21:11:53.713600 24537 heartbeater.cc:511] Master 127.16.178.62:37227 requested a full tablet report, sending...
I20260806 21:11:53.715451 24255 ts_manager.cc:194] Registered new tserver with Master: 94ccbe9c249c4b0eb7209cff25f7928c (127.16.178.1:40607)
W20260806 21:11:53.828384 24541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:53.828696 24541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:53.828738 24541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:53.833015 24541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:53.833096 24541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:53.833155 24541 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.2
I20260806 21:11:53.838239 24541 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:40695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:37227
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24541
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.2
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:53.839453 24541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:53.840910 24541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:53.851497 24551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:53.851502 24552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:53.851449 24556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:53.851707 24541 server_base.cc:1034] running on GCE node
I20260806 21:11:53.852322 24541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:53.853068 24541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:53.854266 24541 hybrid_clock.cc:648] HybridClock initialized: now 1786050713854238 us; error 44 us; skew 500 ppm
I20260806 21:11:53.856527 24541 webserver.cc:460] Webserver started at http://127.16.178.2:33429/ using document root <none> and password file <none>
I20260806 21:11:53.857200 24541 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:53.857278 24541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:53.857517 24541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:53.859848 24541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-1/data/instance:
uuid: "5de6060ddf23425dbcae13e705b8f2b7"
format_stamp: "Formatted at 2026-08-06 21:11:53 on dist-test-slave-lgr5"
server_key: "4a8ab7e525ad0de75bbb9f15de2f7f78"
server_key_iv: "a156aa0f854ca6c7ba05edeac95f8103"
server_key_version: "encryptionkey@0"
I20260806 21:11:53.860538 24541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance:
uuid: "5de6060ddf23425dbcae13e705b8f2b7"
format_stamp: "Formatted at 2026-08-06 21:11:53 on dist-test-slave-lgr5"
server_key: "4a8ab7e525ad0de75bbb9f15de2f7f78"
server_key_iv: "a156aa0f854ca6c7ba05edeac95f8103"
server_key_version: "encryptionkey@0"
I20260806 21:11:53.864564 24541 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:11:53.867350 24565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:53.868934 24541 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:11:53.869091 24541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-1/wal
uuid: "5de6060ddf23425dbcae13e705b8f2b7"
format_stamp: "Formatted at 2026-08-06 21:11:53 on dist-test-slave-lgr5"
server_key: "4a8ab7e525ad0de75bbb9f15de2f7f78"
server_key_iv: "a156aa0f854ca6c7ba05edeac95f8103"
server_key_version: "encryptionkey@0"
I20260806 21:11:53.869235 24541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:53.874369 24541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:53.875116 24541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:53.875300 24541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:53.875892 24541 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:11:53.876935 24541 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:11:53.876986 24541 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:53.877040 24541 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:11:53.877084 24541 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:53.885507 24541 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.2:38787
I20260806 21:11:53.885547 24741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.2:38787 every 8 connection(s)
I20260806 21:11:53.886410 24541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
I20260806 21:11:53.887545 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 24541
I20260806 21:11:53.887662 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance
I20260806 21:11:53.887967 17096 external_mini_cluster.cc:1428] Setting key 60a09dcf0f8727cd7191b53ff4055552
I20260806 21:11:53.890183 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.3:0
--local_ip_for_outbound_sockets=127.16.178.3
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:37227
--builtin_ntp_servers=127.16.178.20:40695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 21:11:53.892856 24318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:53.888420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:54561 (local address 127.16.178.62:37227)
0806 21:11:53.888588 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:53.888592 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:53.889791 (+ 1199us) server_negotiation.cc:407] Connection header received
0806 21:11:53.891194 (+ 1403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:53.891197 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:53.891243 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:53.891315 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:53.892426 (+ 1111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:53.892430 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:53.892433 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:11:53.892446 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:53.892468 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:11:53.892476 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:53.892558 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:53.892678 (+ 120us) server_negotiation.cc:299] Negotiation successful
0806 21:11:53.892723 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 21:11:53.893712 24744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:53.888895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37227 (local address 127.16.178.2:54561)
0806 21:11:53.889656 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:53.889699 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:53.890991 (+ 1292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:53.891482 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:53.891489 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:53.891580 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:53.892167 (+ 587us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 21:11:53.892168 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:53.892210 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 21:11:53.892280 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:53.892546 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:53.892554 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:53.892560 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 21:11:53.892717 (+ 157us) client_negotiation.cc:240] Negotiation successful
0806 21:11:53.892974 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":144,"threads_started":1}
I20260806 21:11:53.894958 24742 heartbeater.cc:347] Connected to a master server at 127.16.178.62:37227
I20260806 21:11:53.895192 24742 heartbeater.cc:464] Registering TS with master...
I20260806 21:11:53.895619 24742 heartbeater.cc:511] Master 127.16.178.62:37227 requested a full tablet report, sending...
I20260806 21:11:53.896710 24255 ts_manager.cc:194] Registered new tserver with Master: 5de6060ddf23425dbcae13e705b8f2b7 (127.16.178.2:38787)
W20260806 21:11:54.009500 24745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:54.009829 24745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:54.009891 24745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:54.014148 24745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:54.014230 24745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:54.014313 24745 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.3
I20260806 21:11:54.019618 24745 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:40695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:37227
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.3
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:54.020896 24745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:54.022367 24745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:54.032014 24753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:54.032130 24751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:54.032196 24745 server_base.cc:1034] running on GCE node
W20260806 21:11:54.032002 24755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:54.032832 24745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:54.033497 24745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:54.034667 24745 hybrid_clock.cc:648] HybridClock initialized: now 1786050714034642 us; error 38 us; skew 500 ppm
I20260806 21:11:54.036616 24745 webserver.cc:460] Webserver started at http://127.16.178.3:40735/ using document root <none> and password file <none>
I20260806 21:11:54.037227 24745 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:54.037298 24745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:54.037515 24745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:54.039698 24745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-2/data/instance:
uuid: "2be7ef91127c4b36974a3d2be1ccea43"
format_stamp: "Formatted at 2026-08-06 21:11:54 on dist-test-slave-lgr5"
server_key: "ac15052b3432ead9d12aac304b5168e8"
server_key_iv: "edf2456a389bd3857c07551de0a8011a"
server_key_version: "encryptionkey@0"
I20260806 21:11:54.040385 24745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance:
uuid: "2be7ef91127c4b36974a3d2be1ccea43"
format_stamp: "Formatted at 2026-08-06 21:11:54 on dist-test-slave-lgr5"
server_key: "ac15052b3432ead9d12aac304b5168e8"
server_key_iv: "edf2456a389bd3857c07551de0a8011a"
server_key_version: "encryptionkey@0"
I20260806 21:11:54.044409 24745 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 21:11:54.047091 24763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:54.048434 24745 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 21:11:54.048568 24745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-2/wal
uuid: "2be7ef91127c4b36974a3d2be1ccea43"
format_stamp: "Formatted at 2026-08-06 21:11:54 on dist-test-slave-lgr5"
server_key: "ac15052b3432ead9d12aac304b5168e8"
server_key_iv: "edf2456a389bd3857c07551de0a8011a"
server_key_version: "encryptionkey@0"
I20260806 21:11:54.048669 24745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:54.053232 24745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:54.053853 24745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:54.053998 24745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:54.054515 24745 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:11:54.055454 24745 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:11:54.055509 24745 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:54.055548 24745 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:11:54.055567 24745 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:54.063895 24745 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.3:39185
I20260806 21:11:54.063917 24931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.3:39185 every 8 connection(s)
I20260806 21:11:54.064854 24745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
I20260806 21:11:54.067818 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 24745
I20260806 21:11:54.067947 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance
I20260806 21:11:54.068354 17096 external_mini_cluster.cc:1428] Setting key 863f2f011e18c0f3fb00861a617b42c2
I20260806 21:11:54.076884 24318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:54.066720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:56243 (local address 127.16.178.62:37227)
0806 21:11:54.066905 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:54.066909 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:54.068163 (+ 1254us) server_negotiation.cc:407] Connection header received
0806 21:11:54.069700 (+ 1537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:54.069703 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:54.069745 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:54.069820 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:54.076381 (+ 6561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:54.076385 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:54.076387 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:11:54.076405 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:54.076433 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:11:54.076442 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:54.076540 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:54.076710 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 21:11:54.076753 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 21:11:54.077891 24934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:54.067112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37227 (local address 127.16.178.3:56243)
0806 21:11:54.067927 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:54.067986 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:54.069489 (+ 1503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:54.069986 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:54.069993 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:54.070081 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:54.075982 (+ 5901us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 21:11:54.075984 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:54.076041 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 21:11:54.076185 (+ 144us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:54.076559 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:54.076569 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:54.076576 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 21:11:54.076749 (+ 173us) client_negotiation.cc:240] Negotiation successful
0806 21:11:54.077009 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":135,"threads_started":1}
I20260806 21:11:54.079078 24932 heartbeater.cc:347] Connected to a master server at 127.16.178.62:37227
I20260806 21:11:54.079312 24932 heartbeater.cc:464] Registering TS with master...
I20260806 21:11:54.079758 24932 heartbeater.cc:511] Master 127.16.178.62:37227 requested a full tablet report, sending...
I20260806 21:11:54.081032 24255 ts_manager.cc:194] Registered new tserver with Master: 2be7ef91127c4b36974a3d2be1ccea43 (127.16.178.3:39185)
I20260806 21:11:54.082715 17096 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 21:11:54.086220 24318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:54.084821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55506 (local address 127.16.178.62:37227)
0806 21:11:54.084956 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:54.084960 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:54.085131 (+ 171us) server_negotiation.cc:407] Connection header received
0806 21:11:54.085209 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:54.085211 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:54.085250 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:54.085318 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:54.085930 (+ 612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:54.085933 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:54.085935 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:11:54.085946 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:54.085966 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:11:54.085975 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:54.086045 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:54.086081 (+ 36us) server_negotiation.cc:299] Negotiation successful
0806 21:11:54.086115 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 21:11:54.089366 24252 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55506:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:11:54.092578 24252 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:11:54.105223 24951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:54.101637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.3:39185 (local address 127.0.0.1:52736)
0806 21:11:54.102752 (+ 1115us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:54.102805 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:54.102917 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:54.103519 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:54.103526 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:54.103611 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:54.104614 (+ 1003us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 21:11:54.104616 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:54.104623 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 21:11:54.104641 (+ 18us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:54.104938 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:54.104941 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:54.104944 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 21:11:54.105009 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 21:11:54.105066 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":933,"thread_start_us":113,"threads_started":1}
I20260806 21:11:54.105285 24953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:54.101989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40890 (local address 127.16.178.1:40607)
0806 21:11:54.102470 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:54.102484 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:54.103037 (+ 553us) server_negotiation.cc:407] Connection header received
0806 21:11:54.103231 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:54.103238 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:54.103383 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:54.103503 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:54.104689 (+ 1186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:54.104695 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:54.104703 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:11:54.104713 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:54.104747 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:11:54.104771 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:54.104869 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:54.105060 (+ 191us) server_negotiation.cc:299] Negotiation successful
0806 21:11:54.105148 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":149,"threads_started":1}
I20260806 21:11:54.105255 24954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:54.102405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52736 (local address 127.16.178.3:39185)
0806 21:11:54.102808 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:54.102819 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:54.103022 (+ 203us) server_negotiation.cc:407] Connection header received
0806 21:11:54.103082 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:54.103088 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:54.103346 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:54.103504 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:54.104759 (+ 1255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:54.104764 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:54.104773 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:11:54.104783 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:54.104829 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:11:54.104865 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:54.104972 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:54.105018 (+ 46us) server_negotiation.cc:299] Negotiation successful
0806 21:11:54.105097 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":103,"threads_started":1}
I20260806 21:11:54.105297 24956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:54.102507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37768 (local address 127.16.178.2:38787)
0806 21:11:54.103011 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:54.103023 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:54.103283 (+ 260us) server_negotiation.cc:407] Connection header received
0806 21:11:54.103332 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:54.103336 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:54.103495 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:54.103616 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:54.104619 (+ 1003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:54.104627 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:54.104641 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:11:54.104655 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:54.104705 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:11:54.104741 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:54.104868 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:54.105107 (+ 239us) server_negotiation.cc:299] Negotiation successful
0806 21:11:54.105165 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":139,"threads_started":1}
I20260806 21:11:54.105824 24952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:54.101934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.2:38787 (local address 127.0.0.1:37768)
0806 21:11:54.103135 (+ 1201us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:54.103147 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:54.103213 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:54.103936 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:54.103939 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:54.103955 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:54.104399 (+ 444us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 21:11:54.104400 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:54.104416 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 21:11:54.104485 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:54.104835 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:54.104854 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:54.104860 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 21:11:54.104972 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 21:11:54.105695 (+ 723us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1148,"mutex_wait_us":625,"thread_start_us":93,"threads_started":1}
I20260806 21:11:54.106406 24949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:54.101718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:40607 (local address 127.0.0.1:40890)
0806 21:11:54.102937 (+ 1219us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:54.102951 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:54.103103 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:54.104186 (+ 1083us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:54.104188 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:54.104201 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:54.104561 (+ 360us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 21:11:54.104563 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:54.104573 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 21:11:54.104597 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:54.104917 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:54.104920 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:54.104924 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 21:11:54.106251 (+ 1327us) client_negotiation.cc:240] Negotiation successful
0806 21:11:54.106288 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1158,"thread_start_us":169,"threads_started":1}
I20260806 21:11:54.108891 24836 tablet_service.cc:1467] Processing CreateTablet for tablet 7b05ea7766054f04a65836bd3b6faf74 (DEFAULT_TABLE table=test-table [id=fbb012e1b7914761b18b20b0b696565c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:11:54.108891 24632 tablet_service.cc:1467] Processing CreateTablet for tablet 7b05ea7766054f04a65836bd3b6faf74 (DEFAULT_TABLE table=test-table [id=fbb012e1b7914761b18b20b0b696565c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:11:54.108875 24440 tablet_service.cc:1467] Processing CreateTablet for tablet 7b05ea7766054f04a65836bd3b6faf74 (DEFAULT_TABLE table=test-table [id=fbb012e1b7914761b18b20b0b696565c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:11:54.110029 24836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b05ea7766054f04a65836bd3b6faf74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:54.110029 24632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b05ea7766054f04a65836bd3b6faf74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:54.110033 24440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b05ea7766054f04a65836bd3b6faf74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:54.116422 24957 tablet_bootstrap.cc:492] T 7b05ea7766054f04a65836bd3b6faf74 P 94ccbe9c249c4b0eb7209cff25f7928c: Bootstrap starting.
I20260806 21:11:54.117895 24960 tablet_bootstrap.cc:492] T 7b05ea7766054f04a65836bd3b6faf74 P 2be7ef91127c4b36974a3d2be1ccea43: Bootstrap starting.
I20260806 21:11:54.117895 24959 tablet_bootstrap.cc:492] T 7b05ea7766054f04a65836bd3b6faf74 P 5de6060ddf23425dbcae13e705b8f2b7: Bootstrap starting.
I20260806 21:11:54.119356 24957 tablet_bootstrap.cc:654] T 7b05ea7766054f04a65836bd3b6faf74 P 94ccbe9c249c4b0eb7209cff25f7928c: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:54.120103 24959 tablet_bootstrap.cc:654] T 7b05ea7766054f04a65836bd3b6faf74 P 5de6060ddf23425dbcae13e705b8f2b7: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:54.120103 24960 tablet_bootstrap.cc:654] T 7b05ea7766054f04a65836bd3b6faf74 P 2be7ef91127c4b36974a3d2be1ccea43: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:54.120152 24957 log.cc:824] T 7b05ea7766054f04a65836bd3b6faf74 P 94ccbe9c249c4b0eb7209cff25f7928c: Log is configured to *not* fsync() on all Append() calls
I20260806 21:11:54.120934 24959 log.cc:824] T 7b05ea7766054f04a65836bd3b6faf74 P 5de6060ddf23425dbcae13e705b8f2b7: Log is configured to *not* fsync() on all Append() calls
I20260806 21:11:54.120934 24960 log.cc:824] T 7b05ea7766054f04a65836bd3b6faf74 P 2be7ef91127c4b36974a3d2be1ccea43: Log is configured to *not* fsync() on all Append() calls
I20260806 21:11:54.122118 24957 tablet_bootstrap.cc:492] T 7b05ea7766054f04a65836bd3b6faf74 P 94ccbe9c249c4b0eb7209cff25f7928c: No bootstrap required, opened a new log
I20260806 21:11:54.122311 24957 ts_tablet_manager.cc:1397] T 7b05ea7766054f04a65836bd3b6faf74 P 94ccbe9c249c4b0eb7209cff25f7928c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 21:11:54.122843 24959 tablet_bootstrap.cc:492] T 7b05ea7766054f04a65836bd3b6faf74 P 5de6060ddf23425dbcae13e705b8f2b7: No bootstrap required, opened a new log
I20260806 21:11:54.122853 24960 tablet_bootstrap.cc:492] T 7b05ea7766054f04a65836bd3b6faf74 P 2be7ef91127c4b36974a3d2be1ccea43: No bootstrap required, opened a new log
I20260806 21:11:54.123026 24960 ts_tablet_manager.cc:1397] T 7b05ea7766054f04a65836bd3b6faf74 P 2be7ef91127c4b36974a3d2be1ccea43: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 21:11:54.123026 24959 ts_tablet_manager.cc:1397] T 7b05ea7766054f04a65836bd3b6faf74 P 5de6060ddf23425dbcae13e705b8f2b7: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 21:11:54.126323 24957 raft_consensus.cc:348] T 7b05ea7766054f04a65836bd3b6faf74 P 94ccbe9c249c4b0eb7209cff25f7928c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2be7ef91127c4b36974a3d2be1ccea43" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 39185 } } peers { permanent_uuid: "94ccbe9c249c4b0eb7209cff25f7928c" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 40607 } } peers { permanent_uuid: "5de6060ddf23425dbcae13e705b8f2b7" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38787 } }
I20260806 21:11:54.126606 24957 raft_consensus.cc:374] T 7b05ea7766054f04a65836bd3b6faf74 P 94ccbe9c249c4b0eb7209cff25f7928c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:54.126665 24957 raft_consensus.cc:729] T 7b05ea7766054f04a65836bd3b6faf74 P 94ccbe9c249c4b0eb7209cff25f7928c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94ccbe9c249c4b0eb7209cff25f7928c, State: Initialized, Role: FOLLOWER
I20260806 21:11:54.127141 24957 consensus_queue.cc:260] T 7b05ea7766054f04a65836bd3b6faf74 P 94ccbe9c249c4b0eb7209cff25f7928c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2be7ef91127c4b36974a3d2be1ccea43" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 39185 } } peers { permanent_uuid: "94ccbe9c249c4b0eb7209cff25f7928c" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 40607 } } peers { permanent_uuid: "5de6060ddf23425dbcae13e705b8f2b7" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38787 } }
I20260806 21:11:54.127386 24959 raft_consensus.cc:348] T 7b05ea7766054f04a65836bd3b6faf74 P 5de6060ddf23425dbcae13e705b8f2b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2be7ef91127c4b36974a3d2be1ccea43" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 39185 } } peers { permanent_uuid: "94ccbe9c249c4b0eb7209cff25f7928c" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 40607 } } peers { permanent_uuid: "5de6060ddf23425dbcae13e705b8f2b7" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38787 } }
I20260806 21:11:54.127378 24960 raft_consensus.cc:348] T 7b05ea7766054f04a65836bd3b6faf74 P 2be7ef91127c4b36974a3d2be1ccea43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2be7ef91127c4b36974a3d2be1ccea43" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 39185 } } peers { permanent_uuid: "94ccbe9c249c4b0eb7209cff25f7928c" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 40607 } } peers { permanent_uuid: "5de6060ddf23425dbcae13e705b8f2b7" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38787 } }
I20260806 21:11:54.127660 24960 raft_consensus.cc:374] T 7b05ea7766054f04a65836bd3b6faf74 P 2be7ef91127c4b36974a3d2be1ccea43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:54.127660 24959 raft_consensus.cc:374] T 7b05ea7766054f04a65836bd3b6faf74 P 5de6060ddf23425dbcae13e705b8f2b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:54.127720 24960 raft_consensus.cc:729] T 7b05ea7766054f04a65836bd3b6faf74 P 2be7ef91127c4b36974a3d2be1ccea43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2be7ef91127c4b36974a3d2be1ccea43, State: Initialized, Role: FOLLOWER
I20260806 21:11:54.127720 24959 raft_consensus.cc:729] T 7b05ea7766054f04a65836bd3b6faf74 P 5de6060ddf23425dbcae13e705b8f2b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5de6060ddf23425dbcae13e705b8f2b7, State: Initialized, Role: FOLLOWER
I20260806 21:11:54.128021 24537 heartbeater.cc:503] Master 127.16.178.62:37227 was elected leader, sending a full tablet report...
I20260806 21:11:54.128257 24960 consensus_queue.cc:260] T 7b05ea7766054f04a65836bd3b6faf74 P 2be7ef91127c4b36974a3d2be1ccea43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2be7ef91127c4b36974a3d2be1ccea43" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 39185 } } peers { permanent_uuid: "94ccbe9c249c4b0eb7209cff25f7928c" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 40607 } } peers { permanent_uuid: "5de6060ddf23425dbcae13e705b8f2b7" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38787 } }
I20260806 21:11:54.128334 24957 ts_tablet_manager.cc:1428] T 7b05ea7766054f04a65836bd3b6faf74 P 94ccbe9c249c4b0eb7209cff25f7928c: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 21:11:54.128257 24959 consensus_queue.cc:260] T 7b05ea7766054f04a65836bd3b6faf74 P 5de6060ddf23425dbcae13e705b8f2b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2be7ef91127c4b36974a3d2be1ccea43" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 39185 } } peers { permanent_uuid: "94ccbe9c249c4b0eb7209cff25f7928c" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 40607 } } peers { permanent_uuid: "5de6060ddf23425dbcae13e705b8f2b7" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38787 } }
I20260806 21:11:54.129266 24960 ts_tablet_manager.cc:1428] T 7b05ea7766054f04a65836bd3b6faf74 P 2be7ef91127c4b36974a3d2be1ccea43: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 21:11:54.129266 24959 ts_tablet_manager.cc:1428] T 7b05ea7766054f04a65836bd3b6faf74 P 5de6060ddf23425dbcae13e705b8f2b7: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 21:11:54.129621 24932 heartbeater.cc:503] Master 127.16.178.62:37227 was elected leader, sending a full tablet report...
I20260806 21:11:54.129717 24742 heartbeater.cc:503] Master 127.16.178.62:37227 was elected leader, sending a full tablet report...
W20260806 21:11:54.137313 24743 tablet.cc:2369] T 7b05ea7766054f04a65836bd3b6faf74 P 5de6060ddf23425dbcae13e705b8f2b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 21:11:54.168937 24964 raft_consensus.cc:482] T 7b05ea7766054f04a65836bd3b6faf74 P 94ccbe9c249c4b0eb7209cff25f7928c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:11:54.169205 24964 raft_consensus.cc:504] T 7b05ea7766054f04a65836bd3b6faf74 P 94ccbe9c249c4b0eb7209cff25f7928c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2be7ef91127c4b36974a3d2be1ccea43" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 39185 } } peers { permanent_uuid: "94ccbe9c249c4b0eb7209cff25f7928c" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 40607 } } peers { permanent_uuid: "5de6060ddf23425dbcae13e705b8f2b7" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38787 } }
I20260806 21:11:54.170440 24964 leader_election.cc:288] T 7b05ea7766054f04a65836bd3b6faf74 P 94ccbe9c249c4b0eb7209cff25f7928c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2be7ef91127c4b36974a3d2be1ccea43 (127.16.178.3:39185), 5de6060ddf23425dbcae13e705b8f2b7 (127.16.178.2:38787)
I20260806 21:11:54.172675 24956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:54.170992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:41683 (local address 127.16.178.2:38787)
0806 21:11:54.171140 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:54.171144 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:54.171160 (+ 16us) server_negotiation.cc:407] Connection header received
0806 21:11:54.171209 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:54.171212 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:54.171244 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:54.171313 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:54.172193 (+ 880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:54.172197 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:54.172199 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:11:54.172214 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:54.172238 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:11:54.172249 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:54.172347 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:54.172528 (+ 181us) server_negotiation.cc:299] Negotiation successful
0806 21:11:54.172566 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 21:11:54.172675 24954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:54.170933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:60367 (local address 127.16.178.3:39185)
0806 21:11:54.171081 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:54.171086 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:54.171399 (+ 313us) server_negotiation.cc:407] Connection header received
0806 21:11:54.171466 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:54.171468 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:54.171502 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:54.171561 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:54.172193 (+ 632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:54.172197 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:54.172200 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:11:54.172214 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:54.172238 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:11:54.172249 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:54.172347 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:54.172485 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 21:11:54.172533 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 21:11:54.172701 24540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:54.170742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.2:38787 (local address 127.16.178.1:41683)
0806 21:11:54.170942 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:54.170964 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:54.171088 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:54.171394 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:54.171397 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:54.171411 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:54.171971 (+ 560us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 21:11:54.171974 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:54.171988 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 21:11:54.172021 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:54.172390 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:54.172393 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:54.172396 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 21:11:54.172477 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 21:11:54.172552 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"mutex_wait_us":26}
I20260806 21:11:54.172701 24967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:54.170742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.3:39185 (local address 127.16.178.1:60367)
0806 21:11:54.171285 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:54.171299 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:54.171396 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:54.171596 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:54.171599 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:54.171614 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:54.171990 (+ 376us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 21:11:54.171991 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:54.171999 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 21:11:54.172021 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:54.172370 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:54.172374 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:54.172379 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 21:11:54.172477 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 21:11:54.172538 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"mutex_wait_us":141,"thread_start_us":155,"threads_started":1}
I20260806 21:11:54.173445 24871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b05ea7766054f04a65836bd3b6faf74" candidate_uuid: "94ccbe9c249c4b0eb7209cff25f7928c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2be7ef91127c4b36974a3d2be1ccea43" is_pre_election: true
I20260806 21:11:54.173445 24666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b05ea7766054f04a65836bd3b6faf74" candidate_uuid: "94ccbe9c249c4b0eb7209cff25f7928c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5de6060ddf23425dbcae13e705b8f2b7" is_pre_election: true
I20260806 21:11:54.173817 24666 raft_consensus.cc:2454] T 7b05ea7766054f04a65836bd3b6faf74 P 5de6060ddf23425dbcae13e705b8f2b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94ccbe9c249c4b0eb7209cff25f7928c in term 0.
I20260806 21:11:54.173822 24871 raft_consensus.cc:2454] T 7b05ea7766054f04a65836bd3b6faf74 P 2be7ef91127c4b36974a3d2be1ccea43 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94ccbe9c249c4b0eb7209cff25f7928c in term 0.
I20260806 21:11:54.174345 24369 leader_election.cc:302] T 7b05ea7766054f04a65836bd3b6faf74 P 94ccbe9c249c4b0eb7209cff25f7928c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2be7ef91127c4b36974a3d2be1ccea43, 94ccbe9c249c4b0eb7209cff25f7928c; no voters:
I20260806 21:11:54.174631 24964 raft_consensus.cc:2781] T 7b05ea7766054f04a65836bd3b6faf74 P 94ccbe9c249c4b0eb7209cff25f7928c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:11:54.174721 24964 raft_consensus.cc:482] T 7b05ea7766054f04a65836bd3b6faf74 P 94ccbe9c249c4b0eb7209cff25f7928c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:11:54.174772 24964 raft_consensus.cc:3037] T 7b05ea7766054f04a65836bd3b6faf74 P 94ccbe9c249c4b0eb7209cff25f7928c [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:54.176159 24964 raft_consensus.cc:504] T 7b05ea7766054f04a65836bd3b6faf74 P 94ccbe9c249c4b0eb7209cff25f7928c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2be7ef91127c4b36974a3d2be1ccea43" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 39185 } } peers { permanent_uuid: "94ccbe9c249c4b0eb7209cff25f7928c" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 40607 } } peers { permanent_uuid: "5de6060ddf23425dbcae13e705b8f2b7" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38787 } }
I20260806 21:11:54.176535 24964 leader_election.cc:288] T 7b05ea7766054f04a65836bd3b6faf74 P 94ccbe9c249c4b0eb7209cff25f7928c [CANDIDATE]: Term 1 election: Requested vote from peers 2be7ef91127c4b36974a3d2be1ccea43 (127.16.178.3:39185), 5de6060ddf23425dbcae13e705b8f2b7 (127.16.178.2:38787)
I20260806 21:11:54.176914 24666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b05ea7766054f04a65836bd3b6faf74" candidate_uuid: "94ccbe9c249c4b0eb7209cff25f7928c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5de6060ddf23425dbcae13e705b8f2b7"
I20260806 21:11:54.177040 24871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b05ea7766054f04a65836bd3b6faf74" candidate_uuid: "94ccbe9c249c4b0eb7209cff25f7928c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2be7ef91127c4b36974a3d2be1ccea43"
I20260806 21:11:54.177055 24666 raft_consensus.cc:3037] T 7b05ea7766054f04a65836bd3b6faf74 P 5de6060ddf23425dbcae13e705b8f2b7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:54.177172 24871 raft_consensus.cc:3037] T 7b05ea7766054f04a65836bd3b6faf74 P 2be7ef91127c4b36974a3d2be1ccea43 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:54.178138 24666 raft_consensus.cc:2454] T 7b05ea7766054f04a65836bd3b6faf74 P 5de6060ddf23425dbcae13e705b8f2b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94ccbe9c249c4b0eb7209cff25f7928c in term 1.
I20260806 21:11:54.178188 24871 raft_consensus.cc:2454] T 7b05ea7766054f04a65836bd3b6faf74 P 2be7ef91127c4b36974a3d2be1ccea43 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94ccbe9c249c4b0eb7209cff25f7928c in term 1.
I20260806 21:11:54.178483 24368 leader_election.cc:302] T 7b05ea7766054f04a65836bd3b6faf74 P 94ccbe9c249c4b0eb7209cff25f7928c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5de6060ddf23425dbcae13e705b8f2b7, 94ccbe9c249c4b0eb7209cff25f7928c; no voters:
I20260806 21:11:54.178671 24964 raft_consensus.cc:2781] T 7b05ea7766054f04a65836bd3b6faf74 P 94ccbe9c249c4b0eb7209cff25f7928c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:11:54.178929 24964 raft_consensus.cc:686] T 7b05ea7766054f04a65836bd3b6faf74 P 94ccbe9c249c4b0eb7209cff25f7928c [term 1 LEADER]: Becoming Leader. State: Replica: 94ccbe9c249c4b0eb7209cff25f7928c, State: Running, Role: LEADER
I20260806 21:11:54.179299 24964 consensus_queue.cc:237] T 7b05ea7766054f04a65836bd3b6faf74 P 94ccbe9c249c4b0eb7209cff25f7928c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2be7ef91127c4b36974a3d2be1ccea43" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 39185 } } peers { permanent_uuid: "94ccbe9c249c4b0eb7209cff25f7928c" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 40607 } } peers { permanent_uuid: "5de6060ddf23425dbcae13e705b8f2b7" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38787 } }
I20260806 21:11:54.182586 24251 catalog_manager.cc:5495] T 7b05ea7766054f04a65836bd3b6faf74 P 94ccbe9c249c4b0eb7209cff25f7928c reported cstate change: term changed from 0 to 1, leader changed from <none> to 94ccbe9c249c4b0eb7209cff25f7928c (127.16.178.1). New cstate: current_term: 1 leader_uuid: "94ccbe9c249c4b0eb7209cff25f7928c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2be7ef91127c4b36974a3d2be1ccea43" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 39185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94ccbe9c249c4b0eb7209cff25f7928c" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 40607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5de6060ddf23425dbcae13e705b8f2b7" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38787 } health_report { overall_health: UNKNOWN } } }
I20260806 21:11:54.205086 24953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:54.203704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40902 (local address 127.16.178.1:40607)
0806 21:11:54.203828 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:54.203832 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:54.203848 (+ 16us) server_negotiation.cc:407] Connection header received
0806 21:11:54.203936 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:54.203939 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:54.204000 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:54.204069 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:54.204711 (+ 642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:54.204714 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:54.204716 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:11:54.204727 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:54.204745 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:11:54.204752 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:54.204822 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:54.204944 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 21:11:54.204979 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260806 21:11:54.205339 24538 tablet.cc:2369] T 7b05ea7766054f04a65836bd3b6faf74 P 94ccbe9c249c4b0eb7209cff25f7928c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 21:11:54.210970 24871 raft_consensus.cc:1261] T 7b05ea7766054f04a65836bd3b6faf74 P 2be7ef91127c4b36974a3d2be1ccea43 [term 1 FOLLOWER]: Refusing update from remote peer 94ccbe9c249c4b0eb7209cff25f7928c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:11:54.210969 24666 raft_consensus.cc:1261] T 7b05ea7766054f04a65836bd3b6faf74 P 5de6060ddf23425dbcae13e705b8f2b7 [term 1 FOLLOWER]: Refusing update from remote peer 94ccbe9c249c4b0eb7209cff25f7928c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:11:54.211730 24964 consensus_queue.cc:1035] T 7b05ea7766054f04a65836bd3b6faf74 P 94ccbe9c249c4b0eb7209cff25f7928c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2be7ef91127c4b36974a3d2be1ccea43" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 39185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:11:54.211915 24968 consensus_queue.cc:1035] T 7b05ea7766054f04a65836bd3b6faf74 P 94ccbe9c249c4b0eb7209cff25f7928c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5de6060ddf23425dbcae13e705b8f2b7" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:11:54.219960 24977 mvcc.cc:205] Tried to move back new op lower bound from 7315663725400989696 to 7315663725280960512. Current Snapshot: MvccSnapshot[applied={T|T < 7315663725400989696}]
I20260806 21:11:54.228298 24255 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55506:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:11:54.228484 24255 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55506:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:11:54.230929 24255 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9269fe44d4364694bdef0821ae8b6346: Sending DeleteTablet for 3 replicas of tablet 7b05ea7766054f04a65836bd3b6faf74
I20260806 21:11:54.231732 24836 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b05ea7766054f04a65836bd3b6faf74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:11:54 UTC) from {username='slave'} at 127.0.0.1:52736
I20260806 21:11:54.231807 24440 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b05ea7766054f04a65836bd3b6faf74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:11:54 UTC) from {username='slave'} at 127.0.0.1:40890
I20260806 21:11:54.231832 24632 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b05ea7766054f04a65836bd3b6faf74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:11:54 UTC) from {username='slave'} at 127.0.0.1:37768
I20260806 21:11:54.232432 24994 tablet_replica.cc:330] T 7b05ea7766054f04a65836bd3b6faf74 P 2be7ef91127c4b36974a3d2be1ccea43: stopping tablet replica
I20260806 21:11:54.232482 24996 tablet_replica.cc:330] T 7b05ea7766054f04a65836bd3b6faf74 P 5de6060ddf23425dbcae13e705b8f2b7: stopping tablet replica
I20260806 21:11:54.232800 24994 raft_consensus.cc:2229] T 7b05ea7766054f04a65836bd3b6faf74 P 2be7ef91127c4b36974a3d2be1ccea43 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:11:54.232960 24996 raft_consensus.cc:2229] T 7b05ea7766054f04a65836bd3b6faf74 P 5de6060ddf23425dbcae13e705b8f2b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:11:54.233076 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 24347
I20260806 21:11:54.233157 24994 raft_consensus.cc:2258] T 7b05ea7766054f04a65836bd3b6faf74 P 2be7ef91127c4b36974a3d2be1ccea43 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:11:54.236454 24996 raft_consensus.cc:2258] T 7b05ea7766054f04a65836bd3b6faf74 P 5de6060ddf23425dbcae13e705b8f2b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:11:54.237360 24994 ts_tablet_manager.cc:1905] T 7b05ea7766054f04a65836bd3b6faf74 P 2be7ef91127c4b36974a3d2be1ccea43: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:11:54.239892 24996 ts_tablet_manager.cc:1905] T 7b05ea7766054f04a65836bd3b6faf74 P 5de6060ddf23425dbcae13e705b8f2b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:11:54.240185 24994 ts_tablet_manager.cc:1918] T 7b05ea7766054f04a65836bd3b6faf74 P 2be7ef91127c4b36974a3d2be1ccea43: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:11:54.240312 24994 log.cc:1196] T 7b05ea7766054f04a65836bd3b6faf74 P 2be7ef91127c4b36974a3d2be1ccea43: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-2/wal/wals/7b05ea7766054f04a65836bd3b6faf74
I20260806 21:11:54.240703 24994 ts_tablet_manager.cc:1939] T 7b05ea7766054f04a65836bd3b6faf74 P 2be7ef91127c4b36974a3d2be1ccea43: Deleting consensus metadata
I20260806 21:11:54.242398 24233 catalog_manager.cc:4841] TS 2be7ef91127c4b36974a3d2be1ccea43 (127.16.178.3:39185): tablet 7b05ea7766054f04a65836bd3b6faf74 (table test-table [id=fbb012e1b7914761b18b20b0b696565c]) successfully deleted
I20260806 21:11:54.242785 24996 ts_tablet_manager.cc:1918] T 7b05ea7766054f04a65836bd3b6faf74 P 5de6060ddf23425dbcae13e705b8f2b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:11:54.242975 24996 log.cc:1196] T 7b05ea7766054f04a65836bd3b6faf74 P 5de6060ddf23425dbcae13e705b8f2b7: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786050686797304-17096-0/minicluster-data/ts-1/wal/wals/7b05ea7766054f04a65836bd3b6faf74
I20260806 21:11:54.243352 24996 ts_tablet_manager.cc:1939] T 7b05ea7766054f04a65836bd3b6faf74 P 5de6060ddf23425dbcae13e705b8f2b7: Deleting consensus metadata
I20260806 21:11:54.244467 24232 catalog_manager.cc:4841] TS 5de6060ddf23425dbcae13e705b8f2b7 (127.16.178.2:38787): tablet 7b05ea7766054f04a65836bd3b6faf74 (table test-table [id=fbb012e1b7914761b18b20b0b696565c]) successfully deleted
W20260806 21:11:54.248998 24234 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.178.1:40607 (error 108)
I20260806 21:11:54.249204 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 24541
I20260806 21:11:54.255229 24952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:54.254648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:40607 (local address 127.0.0.1:40906)
0806 21:11:54.254966 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:54.255103 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.1:40607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186}
W20260806 21:11:54.255512 24234 catalog_manager.cc:4584] TS 94ccbe9c249c4b0eb7209cff25f7928c (127.16.178.1:40607): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7b05ea7766054f04a65836bd3b6faf74: Network error: Client connection negotiation failed: client connection to 127.16.178.1:40607: connect: Connection refused (error 111)
I20260806 21:11:54.273662 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 24745
I20260806 21:11:54.286686 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 24211
2026-08-06T21:11:54Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1085 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260806 21:11:54.315426 17096 mini_oidc.cc:152] Starting JWKS server
I20260806 21:11:54.317291 17096 webserver.cc:460] Webserver started at https://127.0.0.1:43487/ using document root <none> and password file <none>
I20260806 21:11:54.317474 17096 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 21:11:54.317740 17096 webserver.cc:460] Webserver started at http://127.0.0.1:41769/ using document root <none> and password file <none>
2026-08-06T21:11:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:11:54Z Disabled control of system clock
I20260806 21:11:54.327600 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:38591
--webserver_interface=127.16.178.62
--webserver_port=0
--builtin_ntp_servers=127.16.178.20:41889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:38591
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43487/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786050686797304-17096-0/testchainca.cert with env {}
W20260806 21:11:54.440925 25008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:54.441200 25008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:54.441241 25008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:54.445201 25008 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:11:54.445262 25008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:54.445276 25008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:54.445292 25008 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:11:54.445307 25008 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 21:11:54.445319 25008 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 21:11:54.445331 25008 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43487/jwks.json
I20260806 21:11:54.450598 25008 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:41889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:38591
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:38591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43487/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.178.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786050686797304-17096-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25008
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:54.451599 25008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:54.452606 25008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:54.460844 25019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:54.460893 25017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:54.460850 25014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:54.461086 25008 server_base.cc:1034] running on GCE node
I20260806 21:11:54.461665 25008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:54.462805 25008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:54.463981 25008 hybrid_clock.cc:648] HybridClock initialized: now 1786050714463958 us; error 39 us; skew 500 ppm
I20260806 21:11:54.465873 25008 webserver.cc:460] Webserver started at http://127.16.178.62:45709/ using document root <none> and password file <none>
I20260806 21:11:54.466431 25008 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:54.466501 25008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:54.466704 25008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:54.468931 25008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "03775f872c1844e2919c9282d9417304"
format_stamp: "Formatted at 2026-08-06 21:11:54 on dist-test-slave-lgr5"
server_key: "8df8c17b50d445aeb1177c1e5933d763"
server_key_iv: "0882e2356bd385863a954c40c35feda2"
server_key_version: "encryptionkey@0"
I20260806 21:11:54.469497 25008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "03775f872c1844e2919c9282d9417304"
format_stamp: "Formatted at 2026-08-06 21:11:54 on dist-test-slave-lgr5"
server_key: "8df8c17b50d445aeb1177c1e5933d763"
server_key_iv: "0882e2356bd385863a954c40c35feda2"
server_key_version: "encryptionkey@0"
I20260806 21:11:54.473381 25008 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:11:54.475899 25028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:54.477142 25008 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 21:11:54.477270 25008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "03775f872c1844e2919c9282d9417304"
format_stamp: "Formatted at 2026-08-06 21:11:54 on dist-test-slave-lgr5"
server_key: "8df8c17b50d445aeb1177c1e5933d763"
server_key_iv: "0882e2356bd385863a954c40c35feda2"
server_key_version: "encryptionkey@0"
I20260806 21:11:54.477373 25008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:54.488448 25008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:54.489181 25008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:54.489364 25008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:54.496371 25008 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.62:38591
I20260806 21:11:54.496383 25115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.62:38591 every 8 connection(s)
I20260806 21:11:54.497583 25008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:11:54.501389 25117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:54.503584 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 25008
I20260806 21:11:54.503726 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:11:54.504024 17096 external_mini_cluster.cc:1428] Setting key a7d2eb517afe6f849b3d56347319fd49
I20260806 21:11:54.508797 25117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03775f872c1844e2919c9282d9417304: Bootstrap starting.
I20260806 21:11:54.511843 25117 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03775f872c1844e2919c9282d9417304: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:54.512897 25117 log.cc:824] T 00000000000000000000000000000000 P 03775f872c1844e2919c9282d9417304: Log is configured to *not* fsync() on all Append() calls
I20260806 21:11:54.515061 25121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:54.505757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40222 (local address 127.16.178.62:38591)
0806 21:11:54.506494 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:54.506512 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:54.506573 (+ 61us) server_negotiation.cc:407] Connection header received
0806 21:11:54.507483 (+ 910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:54.507507 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:54.507992 (+ 485us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:54.508381 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:54.508806 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:54.509876 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:54.511181 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:54.511719 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:54.511864 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:54.512840 (+ 976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:54.512846 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:54.512862 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:11:54.513354 (+ 492us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:54.513434 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:11:54.513466 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:54.513572 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:54.513742 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 21:11:54.514069 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":139,"threads_started":1}
I20260806 21:11:54.515502 25117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03775f872c1844e2919c9282d9417304: No bootstrap required, opened a new log
I20260806 21:11:54.518615 17096 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:11:54.520387 25117 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03775f872c1844e2919c9282d9417304 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03775f872c1844e2919c9282d9417304" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 38591 } }
I20260806 21:11:54.520753 25117 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03775f872c1844e2919c9282d9417304 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:54.520833 25117 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03775f872c1844e2919c9282d9417304 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03775f872c1844e2919c9282d9417304, State: Initialized, Role: FOLLOWER
I20260806 21:11:54.521373 25117 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03775f872c1844e2919c9282d9417304 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03775f872c1844e2919c9282d9417304" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 38591 } }
I20260806 21:11:54.521529 25117 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03775f872c1844e2919c9282d9417304 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:11:54.521600 25117 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03775f872c1844e2919c9282d9417304 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:11:54.521708 25117 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03775f872c1844e2919c9282d9417304 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:54.522725 25117 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03775f872c1844e2919c9282d9417304 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03775f872c1844e2919c9282d9417304" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 38591 } }
I20260806 21:11:54.523082 25117 leader_election.cc:302] T 00000000000000000000000000000000 P 03775f872c1844e2919c9282d9417304 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03775f872c1844e2919c9282d9417304; no voters:
I20260806 21:11:54.523418 25117 leader_election.cc:288] T 00000000000000000000000000000000 P 03775f872c1844e2919c9282d9417304 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:11:54.523617 25123 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03775f872c1844e2919c9282d9417304 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:11:54.523838 25123 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03775f872c1844e2919c9282d9417304 [term 1 LEADER]: Becoming Leader. State: Replica: 03775f872c1844e2919c9282d9417304, State: Running, Role: LEADER
I20260806 21:11:54.524178 25123 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03775f872c1844e2919c9282d9417304 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03775f872c1844e2919c9282d9417304" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 38591 } }
I20260806 21:11:54.524777 25117 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03775f872c1844e2919c9282d9417304 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:11:54.525837 25124 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03775f872c1844e2919c9282d9417304 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "03775f872c1844e2919c9282d9417304" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03775f872c1844e2919c9282d9417304" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 38591 } } }
I20260806 21:11:54.525980 25124 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03775f872c1844e2919c9282d9417304 [sys.catalog]: This master's current role is: LEADER
I20260806 21:11:54.526372 25136 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:11:54.526625 25125 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03775f872c1844e2919c9282d9417304 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03775f872c1844e2919c9282d9417304. Latest consensus state: current_term: 1 leader_uuid: "03775f872c1844e2919c9282d9417304" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03775f872c1844e2919c9282d9417304" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 38591 } } }
I20260806 21:11:54.526741 25125 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03775f872c1844e2919c9282d9417304 [sys.catalog]: This master's current role is: LEADER
I20260806 21:11:54.529315 25136 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:11:54.534803 25136 catalog_manager.cc:1347] Generated new cluster ID: 1347cd6aae824b0eaa165c9c89bfbff1
I20260806 21:11:54.534889 25136 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:11:54.568140 25136 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:11:54.569193 25136 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:11:54.582572 25136 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03775f872c1844e2919c9282d9417304: Generated new TSK 0
I20260806 21:11:54.583386 25136 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 21:11:54.616381 25168 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260806 21:11:54.616588 25168 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 21:11:54.617146 25121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:54.599472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40226 (local address 127.16.178.62:38591)
0806 21:11:54.599641 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:54.599647 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:54.599892 (+ 245us) server_negotiation.cc:407] Connection header received
0806 21:11:54.600186 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:54.600189 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:54.600247 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:54.600339 (+ 92us) server_negotiation.cc:226] Negotiated authn=JWT
0806 21:11:54.600658 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:54.601229 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:54.602472 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:54.602691 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:54.602765 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:54.603195 (+ 430us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 21:11:54.616718 (+ 13523us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43487/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 21:11:54.617023 (+ 305us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40226: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43487/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":64}
W20260806 21:11:54.617233 25121 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40226: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43487/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260806 21:11:54.618894 25167 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:11:54.599334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:38591 (local address 127.0.0.1:40226)
0806 21:11:54.599788 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:54.599804 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:54.599980 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:54.600378 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:54.600381 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:54.600390 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0806 21:11:54.600531 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:54.600538 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:54.601341 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:54.601345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:54.602347 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:54.602360 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:54.602801 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:54.602805 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:54.602927 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:54.602995 (+ 68us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 21:11:54.617019 (+ 14024us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43487/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 21:11:54.617308 (+ 289us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.16.178.62:38591: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43487/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":149,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.16.178.62:38591: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43487/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260806 21:11:54.619644 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 25008
2026-08-06T21:11:54Z chronyd exiting
I20260806 21:11:54.668699 17096 test_util.cc:175] -----------------------------------------------
I20260806 21:11:54.668794 17096 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786050686797304-17096-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (355 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260806 21:11:54.669917 17096 mini_oidc.cc:152] Starting JWKS server
I20260806 21:11:54.670795 17096 webserver.cc:460] Webserver started at https://127.0.0.1:36789/ using document root <none> and password file <none>
I20260806 21:11:54.670888 17096 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 21:11:54.671146 17096 webserver.cc:460] Webserver started at http://127.0.0.1:43799/ using document root <none> and password file <none>
2026-08-06T21:11:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:11:54Z Disabled control of system clock
I20260806 21:11:54.681139 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:36625
--webserver_interface=127.16.178.62
--webserver_port=0
--builtin_ntp_servers=127.16.178.20:36725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:36625
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36789/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786050686797304-17096-0/testchainca.cert with env {}
W20260806 21:11:54.798085 25175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:54.798381 25175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:54.798422 25175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:54.802362 25175 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:11:54.802418 25175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:54.802433 25175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:54.802448 25175 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:11:54.802460 25175 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 21:11:54.802475 25175 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 21:11:54.802487 25175 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36789/jwks.json
I20260806 21:11:54.807915 25175 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:36725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:36625
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:36625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36789/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.178.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786050686797304-17096-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25175
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:54.808995 25175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:54.809974 25175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:54.819252 25185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:54.819303 25175 server_base.cc:1034] running on GCE node
W20260806 21:11:54.819195 25186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:54.819195 25189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:54.820147 25175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:54.821308 25175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:54.822463 25175 hybrid_clock.cc:648] HybridClock initialized: now 1786050714822441 us; error 37 us; skew 500 ppm
I20260806 21:11:54.824426 25175 webserver.cc:460] Webserver started at http://127.16.178.62:45233/ using document root <none> and password file <none>
I20260806 21:11:54.825038 25175 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:54.825104 25175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:54.825306 25175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:54.827559 25175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "a188dbcc91a741ccab62ae4f0da67a4b"
format_stamp: "Formatted at 2026-08-06 21:11:54 on dist-test-slave-lgr5"
server_key: "572f1eb6da938cda542c0b1ade79b298"
server_key_iv: "5936e494457a6861fac7cd4fd966932b"
server_key_version: "encryptionkey@0"
I20260806 21:11:54.828250 25175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "a188dbcc91a741ccab62ae4f0da67a4b"
format_stamp: "Formatted at 2026-08-06 21:11:54 on dist-test-slave-lgr5"
server_key: "572f1eb6da938cda542c0b1ade79b298"
server_key_iv: "5936e494457a6861fac7cd4fd966932b"
server_key_version: "encryptionkey@0"
I20260806 21:11:54.832299 25175 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:11:54.835076 25200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:54.836536 25175 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:11:54.836683 25175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "a188dbcc91a741ccab62ae4f0da67a4b"
format_stamp: "Formatted at 2026-08-06 21:11:54 on dist-test-slave-lgr5"
server_key: "572f1eb6da938cda542c0b1ade79b298"
server_key_iv: "5936e494457a6861fac7cd4fd966932b"
server_key_version: "encryptionkey@0"
I20260806 21:11:54.836803 25175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:54.866449 25175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:54.867290 25175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:54.867504 25175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:54.874686 25175 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.62:36625
I20260806 21:11:54.874699 25277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.62:36625 every 8 connection(s)
I20260806 21:11:54.875787 25175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:11:54.877915 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 25175
I20260806 21:11:54.878046 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:11:54.878346 17096 external_mini_cluster.cc:1428] Setting key 7d05349cf0b9a6f07e062130f45398b2
I20260806 21:11:54.879757 25278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:54.886441 25278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a188dbcc91a741ccab62ae4f0da67a4b: Bootstrap starting.
I20260806 21:11:54.888784 25281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:54.879932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42176 (local address 127.16.178.62:36625)
0806 21:11:54.880827 (+ 895us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:54.880846 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:54.880888 (+ 42us) server_negotiation.cc:407] Connection header received
0806 21:11:54.881578 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:54.881598 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:54.882091 (+ 493us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:54.882506 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:54.882989 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:54.884024 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:54.884998 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:54.885525 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:54.885691 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:54.886826 (+ 1135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:54.886832 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:54.886847 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:11:54.887269 (+ 422us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:54.887336 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:11:54.887359 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:54.887467 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:54.887624 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 21:11:54.887925 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":190,"threads_started":1}
I20260806 21:11:54.889117 25278 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a188dbcc91a741ccab62ae4f0da67a4b: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:54.889950 25278 log.cc:824] T 00000000000000000000000000000000 P a188dbcc91a741ccab62ae4f0da67a4b: Log is configured to *not* fsync() on all Append() calls
I20260806 21:11:54.892000 17096 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:11:54.892585 25278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a188dbcc91a741ccab62ae4f0da67a4b: No bootstrap required, opened a new log
I20260806 21:11:54.896373 25278 raft_consensus.cc:348] T 00000000000000000000000000000000 P a188dbcc91a741ccab62ae4f0da67a4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a188dbcc91a741ccab62ae4f0da67a4b" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 36625 } }
I20260806 21:11:54.896689 25278 raft_consensus.cc:374] T 00000000000000000000000000000000 P a188dbcc91a741ccab62ae4f0da67a4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:54.896770 25278 raft_consensus.cc:729] T 00000000000000000000000000000000 P a188dbcc91a741ccab62ae4f0da67a4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a188dbcc91a741ccab62ae4f0da67a4b, State: Initialized, Role: FOLLOWER
I20260806 21:11:54.897233 25278 consensus_queue.cc:260] T 00000000000000000000000000000000 P a188dbcc91a741ccab62ae4f0da67a4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a188dbcc91a741ccab62ae4f0da67a4b" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 36625 } }
I20260806 21:11:54.897379 25278 raft_consensus.cc:388] T 00000000000000000000000000000000 P a188dbcc91a741ccab62ae4f0da67a4b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:11:54.897449 25278 raft_consensus.cc:482] T 00000000000000000000000000000000 P a188dbcc91a741ccab62ae4f0da67a4b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:11:54.897547 25278 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a188dbcc91a741ccab62ae4f0da67a4b [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:54.898535 25278 raft_consensus.cc:504] T 00000000000000000000000000000000 P a188dbcc91a741ccab62ae4f0da67a4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a188dbcc91a741ccab62ae4f0da67a4b" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 36625 } }
I20260806 21:11:54.898888 25278 leader_election.cc:302] T 00000000000000000000000000000000 P a188dbcc91a741ccab62ae4f0da67a4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a188dbcc91a741ccab62ae4f0da67a4b; no voters:
I20260806 21:11:54.899224 25278 leader_election.cc:288] T 00000000000000000000000000000000 P a188dbcc91a741ccab62ae4f0da67a4b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:11:54.899361 25283 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a188dbcc91a741ccab62ae4f0da67a4b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:11:54.899595 25283 raft_consensus.cc:686] T 00000000000000000000000000000000 P a188dbcc91a741ccab62ae4f0da67a4b [term 1 LEADER]: Becoming Leader. State: Replica: a188dbcc91a741ccab62ae4f0da67a4b, State: Running, Role: LEADER
I20260806 21:11:54.899947 25283 consensus_queue.cc:237] T 00000000000000000000000000000000 P a188dbcc91a741ccab62ae4f0da67a4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a188dbcc91a741ccab62ae4f0da67a4b" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 36625 } }
I20260806 21:11:54.900540 25278 sys_catalog.cc:564] T 00000000000000000000000000000000 P a188dbcc91a741ccab62ae4f0da67a4b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:11:54.901774 25286 sys_catalog.cc:455] T 00000000000000000000000000000000 P a188dbcc91a741ccab62ae4f0da67a4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader a188dbcc91a741ccab62ae4f0da67a4b. Latest consensus state: current_term: 1 leader_uuid: "a188dbcc91a741ccab62ae4f0da67a4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a188dbcc91a741ccab62ae4f0da67a4b" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 36625 } } }
I20260806 21:11:54.901809 25285 sys_catalog.cc:455] T 00000000000000000000000000000000 P a188dbcc91a741ccab62ae4f0da67a4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a188dbcc91a741ccab62ae4f0da67a4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a188dbcc91a741ccab62ae4f0da67a4b" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 36625 } } }
I20260806 21:11:54.901944 25286 sys_catalog.cc:458] T 00000000000000000000000000000000 P a188dbcc91a741ccab62ae4f0da67a4b [sys.catalog]: This master's current role is: LEADER
I20260806 21:11:54.901947 25285 sys_catalog.cc:458] T 00000000000000000000000000000000 P a188dbcc91a741ccab62ae4f0da67a4b [sys.catalog]: This master's current role is: LEADER
I20260806 21:11:54.902383 25297 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:11:54.905692 25297 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:11:54.912730 25297 catalog_manager.cc:1347] Generated new cluster ID: feee8ce24f70412b8ece928debe1ac43
I20260806 21:11:54.912848 25297 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:11:54.923251 25297 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:11:54.924441 25297 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:11:54.931260 25297 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a188dbcc91a741ccab62ae4f0da67a4b: Generated new TSK 0
I20260806 21:11:54.932430 25297 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 21:11:54.971239 25324 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260806 21:11:54.971349 25324 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 21:11:54.972281 25281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:54.948453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42186 (local address 127.16.178.62:36625)
0806 21:11:54.948622 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:54.948627 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:54.948816 (+ 189us) server_negotiation.cc:407] Connection header received
0806 21:11:54.949006 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:54.949008 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:54.949073 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:54.949174 (+ 101us) server_negotiation.cc:226] Negotiated authn=JWT
0806 21:11:54.949430 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:54.949998 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:54.951271 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:54.951498 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:54.951580 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:54.951979 (+ 399us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 21:11:54.971862 (+ 19883us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36789/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 21:11:54.972155 (+ 293us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42186: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36789/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":58}
W20260806 21:11:54.972376 25281 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42186: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36789/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260806 21:11:54.972559 25323 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:11:54.948357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:36625 (local address 127.0.0.1:42186)
0806 21:11:54.948716 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:54.948732 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:54.948870 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:54.949196 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:54.949199 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:54.949208 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0806 21:11:54.949340 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:54.949346 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:54.950121 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:54.950125 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:54.951128 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:54.951141 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:54.951614 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:54.951618 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:54.951715 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:54.951748 (+ 33us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 21:11:54.972117 (+ 20369us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36789/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 21:11:54.972336 (+ 219us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.16.178.62:36625: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36789/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":110,"threads_started":1}
I20260806 21:11:54.973223 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 25175
2026-08-06T21:11:55Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (355 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260806 21:11:55.025578 17096 mini_oidc.cc:152] Starting JWKS server
I20260806 21:11:55.026554 17096 webserver.cc:460] Webserver started at https://127.0.0.1:41005/ using document root <none> and password file <none>
I20260806 21:11:55.026657 17096 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 21:11:55.026978 17096 webserver.cc:460] Webserver started at http://127.0.0.1:33757/ using document root <none> and password file <none>
2026-08-06T21:11:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:11:55Z Disabled control of system clock
I20260806 21:11:55.037710 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:34433
--webserver_interface=127.16.178.62
--webserver_port=0
--builtin_ntp_servers=127.16.178.20:35223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:34433
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41005/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 21:11:55.154687 25339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:55.155056 25339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:55.155100 25339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:55.159443 25339 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:11:55.159539 25339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:55.159554 25339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:55.159571 25339 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:11:55.159586 25339 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 21:11:55.159601 25339 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 21:11:55.159613 25339 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41005/jwks.json
I20260806 21:11:55.164902 25339 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:35223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:34433
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:34433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41005/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.178.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25339
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:55.166087 25339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:55.167474 25339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:55.176503 25351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:55.176507 25347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:55.176507 25346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:55.176556 25339 server_base.cc:1034] running on GCE node
I20260806 21:11:55.177428 25339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:55.178658 25339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:55.179864 25339 hybrid_clock.cc:648] HybridClock initialized: now 1786050715179845 us; error 30 us; skew 500 ppm
I20260806 21:11:55.182034 25339 webserver.cc:460] Webserver started at http://127.16.178.62:43521/ using document root <none> and password file <none>
I20260806 21:11:55.182698 25339 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:55.182772 25339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:55.182991 25339 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:55.185837 25339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "2f0071b6eea642a2892d9677bfe53e34"
format_stamp: "Formatted at 2026-08-06 21:11:55 on dist-test-slave-lgr5"
server_key: "11fbf06cf0b3b1e93a800d4f0a029a3f"
server_key_iv: "6299e4b3f3c5310e7c7a36f00fbed7eb"
server_key_version: "encryptionkey@0"
I20260806 21:11:55.186575 25339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "2f0071b6eea642a2892d9677bfe53e34"
format_stamp: "Formatted at 2026-08-06 21:11:55 on dist-test-slave-lgr5"
server_key: "11fbf06cf0b3b1e93a800d4f0a029a3f"
server_key_iv: "6299e4b3f3c5310e7c7a36f00fbed7eb"
server_key_version: "encryptionkey@0"
I20260806 21:11:55.191152 25339 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 21:11:55.194087 25358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:55.195662 25339 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:11:55.195837 25339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "2f0071b6eea642a2892d9677bfe53e34"
format_stamp: "Formatted at 2026-08-06 21:11:55 on dist-test-slave-lgr5"
server_key: "11fbf06cf0b3b1e93a800d4f0a029a3f"
server_key_iv: "6299e4b3f3c5310e7c7a36f00fbed7eb"
server_key_version: "encryptionkey@0"
I20260806 21:11:55.196198 25339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:55.206758 25339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:55.207563 25339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:55.207741 25339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:55.214782 25339 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.62:34433
I20260806 21:11:55.214792 25434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.62:34433 every 8 connection(s)
I20260806 21:11:55.215785 25339 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:11:55.219089 25435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:55.224691 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 25339
I20260806 21:11:55.224821 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:11:55.225090 17096 external_mini_cluster.cc:1428] Setting key 3bd1da46da999bc310aa27652028b015
I20260806 21:11:55.225080 25435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f0071b6eea642a2892d9677bfe53e34: Bootstrap starting.
I20260806 21:11:55.227839 25435 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f0071b6eea642a2892d9677bfe53e34: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:55.228672 25435 log.cc:824] T 00000000000000000000000000000000 P 2f0071b6eea642a2892d9677bfe53e34: Log is configured to *not* fsync() on all Append() calls
I20260806 21:11:55.231391 25435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f0071b6eea642a2892d9677bfe53e34: No bootstrap required, opened a new log
I20260806 21:11:55.235651 25440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:55.226617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53796 (local address 127.16.178.62:34433)
0806 21:11:55.227345 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:55.227361 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:55.227400 (+ 39us) server_negotiation.cc:407] Connection header received
0806 21:11:55.228153 (+ 753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:55.228175 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:55.228601 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:55.228936 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:55.229409 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:55.230811 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:55.232042 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:55.232612 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:55.232750 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:55.233700 (+ 950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:55.233706 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:55.233721 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:11:55.234140 (+ 419us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:55.234200 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:11:55.234222 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:55.234303 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:55.234479 (+ 176us) server_negotiation.cc:299] Negotiation successful
0806 21:11:55.234795 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":141,"threads_started":1}
I20260806 21:11:55.235666 25435 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f0071b6eea642a2892d9677bfe53e34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f0071b6eea642a2892d9677bfe53e34" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 34433 } }
I20260806 21:11:55.236004 25435 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f0071b6eea642a2892d9677bfe53e34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:55.236071 25435 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f0071b6eea642a2892d9677bfe53e34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f0071b6eea642a2892d9677bfe53e34, State: Initialized, Role: FOLLOWER
I20260806 21:11:55.236572 25435 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f0071b6eea642a2892d9677bfe53e34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f0071b6eea642a2892d9677bfe53e34" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 34433 } }
I20260806 21:11:55.236730 25435 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f0071b6eea642a2892d9677bfe53e34 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:11:55.236802 25435 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f0071b6eea642a2892d9677bfe53e34 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:11:55.236882 25435 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f0071b6eea642a2892d9677bfe53e34 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:55.237957 25435 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f0071b6eea642a2892d9677bfe53e34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f0071b6eea642a2892d9677bfe53e34" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 34433 } }
I20260806 21:11:55.238353 25435 leader_election.cc:302] T 00000000000000000000000000000000 P 2f0071b6eea642a2892d9677bfe53e34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f0071b6eea642a2892d9677bfe53e34; no voters:
I20260806 21:11:55.238730 25435 leader_election.cc:288] T 00000000000000000000000000000000 P 2f0071b6eea642a2892d9677bfe53e34 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:11:55.238821 25442 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f0071b6eea642a2892d9677bfe53e34 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:11:55.238857 17096 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:11:55.239055 25442 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f0071b6eea642a2892d9677bfe53e34 [term 1 LEADER]: Becoming Leader. State: Replica: 2f0071b6eea642a2892d9677bfe53e34, State: Running, Role: LEADER
I20260806 21:11:55.239398 25442 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f0071b6eea642a2892d9677bfe53e34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f0071b6eea642a2892d9677bfe53e34" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 34433 } }
I20260806 21:11:55.240298 25435 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f0071b6eea642a2892d9677bfe53e34 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:11:55.241321 25443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f0071b6eea642a2892d9677bfe53e34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f0071b6eea642a2892d9677bfe53e34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f0071b6eea642a2892d9677bfe53e34" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 34433 } } }
I20260806 21:11:55.241477 25443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f0071b6eea642a2892d9677bfe53e34 [sys.catalog]: This master's current role is: LEADER
I20260806 21:11:55.241768 25444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f0071b6eea642a2892d9677bfe53e34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f0071b6eea642a2892d9677bfe53e34. Latest consensus state: current_term: 1 leader_uuid: "2f0071b6eea642a2892d9677bfe53e34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f0071b6eea642a2892d9677bfe53e34" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 34433 } } }
I20260806 21:11:55.241870 25444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f0071b6eea642a2892d9677bfe53e34 [sys.catalog]: This master's current role is: LEADER
I20260806 21:11:55.241986 25452 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:11:55.245383 25452 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:11:55.251989 25452 catalog_manager.cc:1347] Generated new cluster ID: 552ac92b6f1b4ca8891f7d9542def8f7
I20260806 21:11:55.252071 25452 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:11:55.263366 25452 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:11:55.264470 25452 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:11:55.271054 25452 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f0071b6eea642a2892d9677bfe53e34: Generated new TSK 0
I20260806 21:11:55.271881 25452 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:11:55.297365 25440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:55.291762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53804 (local address 127.16.178.62:34433)
0806 21:11:55.291903 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:55.291908 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:55.292132 (+ 224us) server_negotiation.cc:407] Connection header received
0806 21:11:55.292333 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:55.292335 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:55.292392 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:55.292490 (+ 98us) server_negotiation.cc:226] Negotiated authn=JWT
0806 21:11:55.292806 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:55.293376 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:55.294657 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:55.294915 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:55.295019 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:55.295363 (+ 344us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 21:11:55.296931 (+ 1568us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41005/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 21:11:55.297186 (+ 255us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53804: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41005/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":40}
W20260806 21:11:55.297458 25440 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53804: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41005/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260806 21:11:55.299185 25483 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:11:55.291639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:34433 (local address 127.0.0.1:53804)
0806 21:11:55.292019 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:55.292035 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:55.292195 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:55.292506 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:55.292509 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:55.292517 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0806 21:11:55.292639 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:55.292645 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:55.293517 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:55.293521 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:55.294508 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:55.294519 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:55.295059 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:55.295063 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:55.295156 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:55.295188 (+ 32us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 21:11:55.297112 (+ 1924us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41005/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 21:11:55.299000 (+ 1888us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.16.178.62:34433: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41005/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":276,"mutex_wait_us":1636,"thread_start_us":104,"threads_started":1}
I20260806 21:11:55.299911 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 25339
E20260806 21:11:55.314131 25484 webserver.cc:533] Webserver: sslize failed:
E20260806 21:11:55.314218 25484 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-06T21:11:55Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (321 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-06T21:11:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:11:55Z Disabled control of system clock
I20260806 21:11:55.357173 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:39959
--webserver_interface=127.16.178.62
--webserver_port=0
--builtin_ntp_servers=127.16.178.20:46285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:39959
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786050686797304-17096-0/testchainca.cert with env {}
W20260806 21:11:55.471647 25494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:55.471943 25494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:55.471984 25494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:55.476070 25494 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:11:55.476197 25494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:55.476212 25494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:55.476228 25494 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:11:55.476249 25494 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 21:11:55.476279 25494 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 21:11:55.476300 25494 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260806 21:11:55.481878 25494 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:46285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:39959
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:39959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.178.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786050686797304-17096-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25494
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:55.483023 25494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:55.484167 25494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:55.493367 25503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:55.493402 25500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:55.493427 25501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:55.493752 25494 server_base.cc:1034] running on GCE node
I20260806 21:11:55.494338 25494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:55.495538 25494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:55.496732 25494 hybrid_clock.cc:648] HybridClock initialized: now 1786050715496692 us; error 54 us; skew 500 ppm
I20260806 21:11:55.498713 25494 webserver.cc:460] Webserver started at http://127.16.178.62:34351/ using document root <none> and password file <none>
I20260806 21:11:55.499289 25494 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:55.499351 25494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:55.499531 25494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:55.501732 25494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "49b65f516d484312ba5d373cd4b3579c"
format_stamp: "Formatted at 2026-08-06 21:11:55 on dist-test-slave-lgr5"
server_key: "4e52ef94916ceaa59137007360a4ec85"
server_key_iv: "55f5104719f719ce9b8460f12702b21d"
server_key_version: "encryptionkey@0"
I20260806 21:11:55.502271 25494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "49b65f516d484312ba5d373cd4b3579c"
format_stamp: "Formatted at 2026-08-06 21:11:55 on dist-test-slave-lgr5"
server_key: "4e52ef94916ceaa59137007360a4ec85"
server_key_iv: "55f5104719f719ce9b8460f12702b21d"
server_key_version: "encryptionkey@0"
I20260806 21:11:55.506167 25494 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:11:55.508524 25510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:55.509768 25494 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 21:11:55.509902 25494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "49b65f516d484312ba5d373cd4b3579c"
format_stamp: "Formatted at 2026-08-06 21:11:55 on dist-test-slave-lgr5"
server_key: "4e52ef94916ceaa59137007360a4ec85"
server_key_iv: "55f5104719f719ce9b8460f12702b21d"
server_key_version: "encryptionkey@0"
I20260806 21:11:55.510008 25494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:55.522055 25494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:55.522739 25494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:55.522959 25494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:55.530684 25494 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.62:39959
I20260806 21:11:55.530702 25598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.62:39959 every 8 connection(s)
I20260806 21:11:55.531793 25494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:11:55.533347 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 25494
I20260806 21:11:55.533468 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:11:55.533766 17096 external_mini_cluster.cc:1428] Setting key 6478c5bebb46c08fbb1d2a594a8ec6af
I20260806 21:11:55.535431 25599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:55.542207 25599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49b65f516d484312ba5d373cd4b3579c: Bootstrap starting.
I20260806 21:11:55.543597 25602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:55.535173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56336 (local address 127.16.178.62:39959)
0806 21:11:55.535826 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:55.535844 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:55.535886 (+ 42us) server_negotiation.cc:407] Connection header received
0806 21:11:55.536692 (+ 806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:55.536710 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:55.537121 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:55.537430 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:55.537799 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:55.538822 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:55.540017 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:55.540606 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:55.540743 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:55.541691 (+ 948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:55.541696 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:55.541710 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:11:55.542138 (+ 428us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:55.542202 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:11:55.542227 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:55.542339 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:55.542507 (+ 168us) server_negotiation.cc:299] Negotiation successful
0806 21:11:55.542833 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":125,"threads_started":1}
I20260806 21:11:55.544960 25599 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49b65f516d484312ba5d373cd4b3579c: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:55.545814 25599 log.cc:824] T 00000000000000000000000000000000 P 49b65f516d484312ba5d373cd4b3579c: Log is configured to *not* fsync() on all Append() calls
I20260806 21:11:55.546897 17096 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:11:55.548365 25599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49b65f516d484312ba5d373cd4b3579c: No bootstrap required, opened a new log
I20260806 21:11:55.552352 25599 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49b65f516d484312ba5d373cd4b3579c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49b65f516d484312ba5d373cd4b3579c" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 39959 } }
I20260806 21:11:55.552680 25599 raft_consensus.cc:374] T 00000000000000000000000000000000 P 49b65f516d484312ba5d373cd4b3579c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:55.552721 25599 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49b65f516d484312ba5d373cd4b3579c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49b65f516d484312ba5d373cd4b3579c, State: Initialized, Role: FOLLOWER
I20260806 21:11:55.553197 25599 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49b65f516d484312ba5d373cd4b3579c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49b65f516d484312ba5d373cd4b3579c" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 39959 } }
I20260806 21:11:55.553340 25599 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49b65f516d484312ba5d373cd4b3579c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:11:55.553395 25599 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49b65f516d484312ba5d373cd4b3579c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:11:55.553514 25599 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49b65f516d484312ba5d373cd4b3579c [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:55.554591 25599 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49b65f516d484312ba5d373cd4b3579c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49b65f516d484312ba5d373cd4b3579c" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 39959 } }
I20260806 21:11:55.554971 25599 leader_election.cc:302] T 00000000000000000000000000000000 P 49b65f516d484312ba5d373cd4b3579c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49b65f516d484312ba5d373cd4b3579c; no voters:
I20260806 21:11:55.555349 25599 leader_election.cc:288] T 00000000000000000000000000000000 P 49b65f516d484312ba5d373cd4b3579c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:11:55.555454 25605 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49b65f516d484312ba5d373cd4b3579c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:11:55.555634 25605 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49b65f516d484312ba5d373cd4b3579c [term 1 LEADER]: Becoming Leader. State: Replica: 49b65f516d484312ba5d373cd4b3579c, State: Running, Role: LEADER
I20260806 21:11:55.555961 25605 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49b65f516d484312ba5d373cd4b3579c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49b65f516d484312ba5d373cd4b3579c" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 39959 } }
I20260806 21:11:55.556898 25599 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49b65f516d484312ba5d373cd4b3579c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:11:55.557955 25611 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49b65f516d484312ba5d373cd4b3579c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49b65f516d484312ba5d373cd4b3579c. Latest consensus state: current_term: 1 leader_uuid: "49b65f516d484312ba5d373cd4b3579c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49b65f516d484312ba5d373cd4b3579c" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 39959 } } }
I20260806 21:11:55.558017 25608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49b65f516d484312ba5d373cd4b3579c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "49b65f516d484312ba5d373cd4b3579c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49b65f516d484312ba5d373cd4b3579c" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 39959 } } }
I20260806 21:11:55.558117 25611 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49b65f516d484312ba5d373cd4b3579c [sys.catalog]: This master's current role is: LEADER
I20260806 21:11:55.558151 25608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49b65f516d484312ba5d373cd4b3579c [sys.catalog]: This master's current role is: LEADER
I20260806 21:11:55.558523 25623 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:11:55.562254 25623 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:11:55.568614 25623 catalog_manager.cc:1347] Generated new cluster ID: b8169d5451514a39bcac72f56e87e6e6
I20260806 21:11:55.568704 25623 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:11:55.600980 25623 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:11:55.602152 25623 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:11:55.611294 25623 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 49b65f516d484312ba5d373cd4b3579c: Generated new TSK 0
I20260806 21:11:55.612217 25623 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:11:56.529188 25602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:55.633946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56350 (local address 127.16.178.62:39959)
0806 21:11:55.634111 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:55.634116 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:55.634222 (+ 106us) server_negotiation.cc:407] Connection header received
0806 21:11:55.634412 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:55.634416 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:55.634482 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:55.634625 (+ 143us) server_negotiation.cc:226] Negotiated authn=JWT
0806 21:11:55.634835 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:55.635652 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:55.636638 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:55.637005 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:55.637113 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:55.637429 (+ 316us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 21:11:56.528762 (+891333us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 21:11:56.529059 (+ 297us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56350: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":76}
W20260806 21:11:56.529274 25602 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56350: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260806 21:11:56.529453 25652 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:11:55.633806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:39959 (local address 127.0.0.1:56350)
0806 21:11:55.634127 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:55.634142 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:55.634259 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:55.634618 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:55.634620 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:55.634627 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0806 21:11:55.634734 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:55.634739 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:55.635806 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:55.635809 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:55.636497 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:55.636528 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:55.637129 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:55.637132 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:55.637211 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:55.637240 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 21:11:56.529018 (+891778us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 21:11:56.529221 (+ 203us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.16.178.62:39959: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":113,"threads_started":1}
I20260806 21:11:56.530035 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 25494
2026-08-06T21:11:56Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1206 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (133 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (129 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:11:56 dist-test-slave-lgr5 krb5kdc[25667](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:11:56 dist-test-slave-lgr5 krb5kdc[25667](info): set up 2 sockets
Aug 06 21:11:56 dist-test-slave-lgr5 krb5kdc[25667](info): commencing operation
krb5kdc: starting...
W20260806 21:11:57.953805 17096 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Aug 06 21:11:58 dist-test-slave-lgr5 krb5kdc[25667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:11:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:11:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.178.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
I20260806 21:11:58.089072 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:34843
--webserver_interface=127.16.178.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:44883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:34843
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:11:58.203410 25693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:58.203723 25693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:58.203766 25693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:58.207836 25693 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:11:58.207906 25693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:58.207924 25693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:58.207939 25693 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:11:58.207953 25693 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:11:58.213380 25693 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:44883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:34843
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:34843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.178.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:58.214561 25693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:58.215629 25693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:58.224673 25704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:58.224673 25706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:58.224687 25702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:58.224802 25693 server_base.cc:1034] running on GCE node
I20260806 21:11:58.225672 25693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:58.226887 25693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:58.228129 25693 hybrid_clock.cc:648] HybridClock initialized: now 1786050718228066 us; error 42 us; skew 500 ppm
Aug 06 21:11:58 dist-test-slave-lgr5 krb5kdc[25667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050718, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:58.231261 25693 init.cc:377] Logged in from keytab as kudu/127.16.178.62@KRBTEST.COM (short username kudu)
I20260806 21:11:58.232388 25693 webserver.cc:460] Webserver started at http://127.16.178.62:46699/ using document root <none> and password file <none>
I20260806 21:11:58.233009 25693 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:58.233083 25693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:58.233299 25693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:58.235549 25693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "3c962a693ec2411c9e0e68a836016acb"
format_stamp: "Formatted at 2026-08-06 21:11:58 on dist-test-slave-lgr5"
server_key: "eadbd361bc92773be019b923b04b00e6"
server_key_iv: "09378704cdb80a4af6718d4de58e1b1d"
server_key_version: "encryptionkey@0"
I20260806 21:11:58.236188 25693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "3c962a693ec2411c9e0e68a836016acb"
format_stamp: "Formatted at 2026-08-06 21:11:58 on dist-test-slave-lgr5"
server_key: "eadbd361bc92773be019b923b04b00e6"
server_key_iv: "09378704cdb80a4af6718d4de58e1b1d"
server_key_version: "encryptionkey@0"
I20260806 21:11:58.240556 25693 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 21:11:58.243440 25716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:58.244887 25693 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:11:58.245054 25693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "3c962a693ec2411c9e0e68a836016acb"
format_stamp: "Formatted at 2026-08-06 21:11:58 on dist-test-slave-lgr5"
server_key: "eadbd361bc92773be019b923b04b00e6"
server_key_iv: "09378704cdb80a4af6718d4de58e1b1d"
server_key_version: "encryptionkey@0"
I20260806 21:11:58.245180 25693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:58.269028 25693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:58.272451 25693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:58.272650 25693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:58.279965 25693 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.62:34843
I20260806 21:11:58.280014 25795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.62:34843 every 8 connection(s)
I20260806 21:11:58.281051 25693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:11:58.284124 25796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:11:58.286319 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 25693
I20260806 21:11:58.286458 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:11:58.286804 17096 external_mini_cluster.cc:1428] Setting key c0f1f94b96b85d11ca3393099a612acc
I20260806 21:11:58.291548 25796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c962a693ec2411c9e0e68a836016acb: Bootstrap starting.
I20260806 21:11:58.294404 25796 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c962a693ec2411c9e0e68a836016acb: Neither blocks nor log segments found. Creating new log.
I20260806 21:11:58.295392 25796 log.cc:824] T 00000000000000000000000000000000 P 3c962a693ec2411c9e0e68a836016acb: Log is configured to *not* fsync() on all Append() calls
Aug 06 21:11:58 dist-test-slave-lgr5 krb5kdc[25667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:11:58.297720 25796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c962a693ec2411c9e0e68a836016acb: No bootstrap required, opened a new log
I20260806 21:11:58.301352 25796 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c962a693ec2411c9e0e68a836016acb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c962a693ec2411c9e0e68a836016acb" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 34843 } }
I20260806 21:11:58.301541 25803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:58.288372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50298 (local address 127.16.178.62:34843)
0806 21:11:58.288811 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:58.288827 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:58.288866 (+ 39us) server_negotiation.cc:407] Connection header received
0806 21:11:58.289619 (+ 753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:58.289640 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:58.289999 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:58.290288 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:58.291242 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:58.292348 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:58.293549 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:58.294039 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:58.294157 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:58.297085 (+ 2928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:58.297090 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:58.297105 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:58.297133 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:58.298744 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:58.299174 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:58.299179 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:58.299184 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:58.299296 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:58.299565 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:58.299567 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:58.299568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:58.299967 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:58.300192 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:58.300478 (+ 286us) server_negotiation.cc:299] Negotiation successful
0806 21:11:58.300777 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":119,"threads_started":1}
I20260806 21:11:58.301606 25796 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c962a693ec2411c9e0e68a836016acb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:11:58.301648 25796 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c962a693ec2411c9e0e68a836016acb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c962a693ec2411c9e0e68a836016acb, State: Initialized, Role: FOLLOWER
I20260806 21:11:58.302105 25796 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c962a693ec2411c9e0e68a836016acb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c962a693ec2411c9e0e68a836016acb" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 34843 } }
I20260806 21:11:58.302241 25796 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c962a693ec2411c9e0e68a836016acb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:11:58.302299 25796 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c962a693ec2411c9e0e68a836016acb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:11:58.302400 25796 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c962a693ec2411c9e0e68a836016acb [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:11:58.303555 25796 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c962a693ec2411c9e0e68a836016acb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c962a693ec2411c9e0e68a836016acb" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 34843 } }
I20260806 21:11:58.303921 25796 leader_election.cc:302] T 00000000000000000000000000000000 P 3c962a693ec2411c9e0e68a836016acb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c962a693ec2411c9e0e68a836016acb; no voters:
I20260806 21:11:58.304288 25796 leader_election.cc:288] T 00000000000000000000000000000000 P 3c962a693ec2411c9e0e68a836016acb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:11:58.304392 25805 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c962a693ec2411c9e0e68a836016acb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:11:58.304620 25805 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c962a693ec2411c9e0e68a836016acb [term 1 LEADER]: Becoming Leader. State: Replica: 3c962a693ec2411c9e0e68a836016acb, State: Running, Role: LEADER
I20260806 21:11:58.304922 25805 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c962a693ec2411c9e0e68a836016acb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c962a693ec2411c9e0e68a836016acb" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 34843 } }
I20260806 21:11:58.305462 25796 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c962a693ec2411c9e0e68a836016acb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:11:58.306481 25810 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c962a693ec2411c9e0e68a836016acb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c962a693ec2411c9e0e68a836016acb. Latest consensus state: current_term: 1 leader_uuid: "3c962a693ec2411c9e0e68a836016acb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c962a693ec2411c9e0e68a836016acb" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 34843 } } }
I20260806 21:11:58.306612 25810 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c962a693ec2411c9e0e68a836016acb [sys.catalog]: This master's current role is: LEADER
I20260806 21:11:58.306888 25807 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c962a693ec2411c9e0e68a836016acb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c962a693ec2411c9e0e68a836016acb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c962a693ec2411c9e0e68a836016acb" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 34843 } } }
I20260806 21:11:58.306982 25807 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c962a693ec2411c9e0e68a836016acb [sys.catalog]: This master's current role is: LEADER
I20260806 21:11:58.307067 25820 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:11:58.310017 25820 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:11:58.315795 25820 catalog_manager.cc:1347] Generated new cluster ID: 6084a9f825d54116a7a34cb42ce43a0b
I20260806 21:11:58.315874 25820 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:11:58.321713 25820 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:11:58.322835 25820 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:11:58.332561 25820 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c962a693ec2411c9e0e68a836016acb: Generated new TSK 0
I20260806 21:11:58.333189 25820 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.178.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab.
Entry for principal kudu/127.16.178.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab.
I20260806 21:11:58.364538 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.1:0
--local_ip_for_outbound_sockets=127.16.178.1
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:34843
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab
--principal=kudu/127.16.178.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:44883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:11:58.482995 25836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:58.483273 25836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:58.483316 25836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:58.487319 25836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:58.487392 25836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:58.487452 25836 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.1
I20260806 21:11:58.492821 25836 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:44883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab
--principal=kudu/127.16.178.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:34843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.1
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:58.493860 25836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:58.494849 25836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:58.504416 25849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:58.504480 25845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:58.504449 25853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:58.504616 25836 server_base.cc:1034] running on GCE node
I20260806 21:11:58.505046 25836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:58.505789 25836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:58.506944 25836 hybrid_clock.cc:648] HybridClock initialized: now 1786050718506913 us; error 50 us; skew 500 ppm
Aug 06 21:11:58 dist-test-slave-lgr5 krb5kdc[25667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050718, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:58.509815 25836 init.cc:377] Logged in from keytab as kudu/127.16.178.1@KRBTEST.COM (short username kudu)
I20260806 21:11:58.510679 25836 webserver.cc:460] Webserver started at http://127.16.178.1:45233/ using document root <none> and password file <none>
I20260806 21:11:58.511255 25836 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:58.511315 25836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:58.511492 25836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:58.513639 25836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-0/data/instance:
uuid: "4269d3385fc341f9a6665abad4eb0559"
format_stamp: "Formatted at 2026-08-06 21:11:58 on dist-test-slave-lgr5"
server_key: "51b87ea939ec42b98a96e7ef832629a2"
server_key_iv: "d56da34647965a81da98427cdbefa9a1"
server_key_version: "encryptionkey@0"
I20260806 21:11:58.514318 25836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance:
uuid: "4269d3385fc341f9a6665abad4eb0559"
format_stamp: "Formatted at 2026-08-06 21:11:58 on dist-test-slave-lgr5"
server_key: "51b87ea939ec42b98a96e7ef832629a2"
server_key_iv: "d56da34647965a81da98427cdbefa9a1"
server_key_version: "encryptionkey@0"
I20260806 21:11:58.518503 25836 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:11:58.521335 25861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:58.522743 25836 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:11:58.522876 25836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-0/wal
uuid: "4269d3385fc341f9a6665abad4eb0559"
format_stamp: "Formatted at 2026-08-06 21:11:58 on dist-test-slave-lgr5"
server_key: "51b87ea939ec42b98a96e7ef832629a2"
server_key_iv: "d56da34647965a81da98427cdbefa9a1"
server_key_version: "encryptionkey@0"
I20260806 21:11:58.523021 25836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:58.534866 25836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:58.538431 25836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:58.538633 25836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:58.539302 25836 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:11:58.540246 25836 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:11:58.540294 25836 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:58.540337 25836 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:11:58.540364 25836 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:58.549077 25836 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.1:40733
I20260806 21:11:58.549136 26038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.1:40733 every 8 connection(s)
I20260806 21:11:58.550064 25836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
I20260806 21:11:58.551985 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 25836
I20260806 21:11:58.552148 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance
I20260806 21:11:58.552464 17096 external_mini_cluster.cc:1428] Setting key 7b92548313c66893a0bccdc5a90c0388
Aug 06 21:11:58 dist-test-slave-lgr5 krb5kdc[25667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050718, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.178.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.2@KRBTEST.COM" created.
I20260806 21:11:58.566216 25803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:58.553187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:53059 (local address 127.16.178.62:34843)
0806 21:11:58.553317 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:58.553321 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:58.554489 (+ 1168us) server_negotiation.cc:407] Connection header received
0806 21:11:58.556061 (+ 1572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:58.556064 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:58.556157 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:58.556236 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:58.557556 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:58.558283 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:58.559609 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:58.559974 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:58.560115 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:58.562831 (+ 2716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:58.562835 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:58.562838 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:58.562868 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:58.564278 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:58.564772 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:58.564774 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:58.564775 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:58.564848 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:58.565114 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:58.565116 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:58.565117 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:58.565320 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:58.565391 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:58.565886 (+ 495us) server_negotiation.cc:299] Negotiation successful
0806 21:11:58.566043 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 21:11:58.567068 26041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:58.553749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:34843 (local address 127.16.178.1:53059)
0806 21:11:58.554329 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:58.554387 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:58.555797 (+ 1410us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:58.556458 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:58.556466 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:58.556865 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:58.557387 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:58.557402 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:58.558432 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:58.558435 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:58.559459 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:58.559471 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:58.560181 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:58.560185 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:58.560372 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:58.560961 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:58.560985 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:58.562635 (+ 1650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:58.564408 (+ 1773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:58.564415 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:58.564433 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:58.564688 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:58.564919 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:58.564921 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:58.564922 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:58.565050 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:58.565411 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:58.565419 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:58.565676 (+ 257us) client_negotiation.cc:769] Sending connection context
0806 21:11:58.565840 (+ 164us) client_negotiation.cc:240] Negotiation successful
0806 21:11:58.566151 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":131,"threads_started":1}
I20260806 21:11:58.568483 26039 heartbeater.cc:347] Connected to a master server at 127.16.178.62:34843
I20260806 21:11:58.568768 26039 heartbeater.cc:464] Registering TS with master...
I20260806 21:11:58.569398 26039 heartbeater.cc:511] Master 127.16.178.62:34843 requested a full tablet report, sending...
I20260806 21:11:58.571244 25743 ts_manager.cc:194] Registered new tserver with Master: 4269d3385fc341f9a6665abad4eb0559 (127.16.178.1:40733)
I20260806 21:11:58.572877 25743 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.1@KRBTEST.COM'} at 127.16.178.1:53059
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab.
Entry for principal kudu/127.16.178.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab.
I20260806 21:11:58.584124 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.2:0
--local_ip_for_outbound_sockets=127.16.178.2
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:34843
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab
--principal=kudu/127.16.178.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:44883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:11:58.700376 26046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:58.700677 26046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:58.700721 26046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:58.705096 26046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:58.705180 26046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:58.705255 26046 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.2
I20260806 21:11:58.711054 26046 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:44883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab
--principal=kudu/127.16.178.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:34843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.2
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:58.712332 26046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:58.713495 26046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:58.724224 26051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:58.724224 26053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:58.724226 26055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:58.724339 26046 server_base.cc:1034] running on GCE node
I20260806 21:11:58.725023 26046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:58.725816 26046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:58.727002 26046 hybrid_clock.cc:648] HybridClock initialized: now 1786050718726976 us; error 44 us; skew 500 ppm
Aug 06 21:11:58 dist-test-slave-lgr5 krb5kdc[25667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050718, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:58.730146 26046 init.cc:377] Logged in from keytab as kudu/127.16.178.2@KRBTEST.COM (short username kudu)
I20260806 21:11:58.731287 26046 webserver.cc:460] Webserver started at http://127.16.178.2:38885/ using document root <none> and password file <none>
I20260806 21:11:58.731902 26046 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:58.731980 26046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:58.732237 26046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:58.734555 26046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-1/data/instance:
uuid: "314e3904274644318b4c3d60a683cdbd"
format_stamp: "Formatted at 2026-08-06 21:11:58 on dist-test-slave-lgr5"
server_key: "b81d8f62ca3881cd18720db24cc9a4cc"
server_key_iv: "305e9993883d7c85e0cfb66c4cd20c6a"
server_key_version: "encryptionkey@0"
I20260806 21:11:58.735165 26046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance:
uuid: "314e3904274644318b4c3d60a683cdbd"
format_stamp: "Formatted at 2026-08-06 21:11:58 on dist-test-slave-lgr5"
server_key: "b81d8f62ca3881cd18720db24cc9a4cc"
server_key_iv: "305e9993883d7c85e0cfb66c4cd20c6a"
server_key_version: "encryptionkey@0"
I20260806 21:11:58.739226 26046 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 21:11:58.741915 26064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:58.743242 26046 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:11:58.743377 26046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-1/wal
uuid: "314e3904274644318b4c3d60a683cdbd"
format_stamp: "Formatted at 2026-08-06 21:11:58 on dist-test-slave-lgr5"
server_key: "b81d8f62ca3881cd18720db24cc9a4cc"
server_key_iv: "305e9993883d7c85e0cfb66c4cd20c6a"
server_key_version: "encryptionkey@0"
I20260806 21:11:58.743500 26046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:58.764284 26046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:58.767568 26046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:58.767761 26046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:58.768427 26046 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:11:58.769323 26046 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:11:58.769372 26046 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:58.769428 26046 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:11:58.769454 26046 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:58.778455 26046 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.2:43695
I20260806 21:11:58.778488 26240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.2:43695 every 8 connection(s)
I20260806 21:11:58.779438 26046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
I20260806 21:11:58.781870 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 26046
I20260806 21:11:58.782001 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance
I20260806 21:11:58.782351 17096 external_mini_cluster.cc:1428] Setting key 9237a548e012abe73258279866e38ee6
Aug 06 21:11:58 dist-test-slave-lgr5 krb5kdc[25667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050718, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.2@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.178.3@KRBTEST.COM; defaulting to no policy
I20260806 21:11:58.795925 25803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:58.781438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:50187 (local address 127.16.178.62:34843)
0806 21:11:58.781589 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:58.781593 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:58.782556 (+ 963us) server_negotiation.cc:407] Connection header received
0806 21:11:58.784143 (+ 1587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:58.784146 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:58.784192 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:58.784260 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:58.785882 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:58.786668 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:58.788013 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:58.788337 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:58.788442 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:58.791578 (+ 3136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:58.791584 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:58.791587 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:58.791627 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:58.793344 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:58.794066 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:58.794070 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:58.794071 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:58.794164 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:58.794634 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:58.794638 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:58.794639 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:58.794882 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:58.794969 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:58.795593 (+ 624us) server_negotiation.cc:299] Negotiation successful
0806 21:11:58.795752 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.3@KRBTEST.COM" created.
I20260806 21:11:58.796969 26244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:58.781812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:34843 (local address 127.16.178.2:50187)
0806 21:11:58.782354 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:58.782420 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:58.783879 (+ 1459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:58.784462 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:58.784482 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:58.784965 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:58.785710 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:58.785732 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:58.786831 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:58.786834 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:58.787846 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:58.787860 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:58.788532 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:58.788536 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:58.788729 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:58.789355 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:58.789375 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:58.791306 (+ 1931us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:58.793541 (+ 2235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:58.793547 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:58.793566 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:58.793950 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:58.794259 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:58.794262 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:58.794263 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:58.794452 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:58.795010 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:58.795016 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:58.795330 (+ 314us) client_negotiation.cc:769] Sending connection context
0806 21:11:58.795531 (+ 201us) client_negotiation.cc:240] Negotiation successful
0806 21:11:58.795890 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":138,"threads_started":1}
I20260806 21:11:58.798302 26241 heartbeater.cc:347] Connected to a master server at 127.16.178.62:34843
I20260806 21:11:58.798614 26241 heartbeater.cc:464] Registering TS with master...
I20260806 21:11:58.799229 26241 heartbeater.cc:511] Master 127.16.178.62:34843 requested a full tablet report, sending...
I20260806 21:11:58.800544 25743 ts_manager.cc:194] Registered new tserver with Master: 314e3904274644318b4c3d60a683cdbd (127.16.178.2:43695)
I20260806 21:11:58.801297 25743 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.2@KRBTEST.COM'} at 127.16.178.2:50187
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab.
Entry for principal kudu/127.16.178.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab.
I20260806 21:11:58.815887 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.3:0
--local_ip_for_outbound_sockets=127.16.178.3
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:34843
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab
--principal=kudu/127.16.178.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:44883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:11:58.932427 26249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:11:58.932694 26249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:11:58.932737 26249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:11:58.936749 26249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:11:58.936800 26249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:11:58.936856 26249 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.3
I20260806 21:11:58.942415 26249 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:44883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab
--principal=kudu/127.16.178.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:34843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26249
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.3
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:11:58.943426 26249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:11:58.944453 26249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:11:58.954402 26263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:58.954401 26257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:11:58.954617 26260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:11:58.954663 26249 server_base.cc:1034] running on GCE node
I20260806 21:11:58.955169 26249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:11:58.955838 26249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:11:58.957006 26249 hybrid_clock.cc:648] HybridClock initialized: now 1786050718956973 us; error 45 us; skew 500 ppm
Aug 06 21:11:58 dist-test-slave-lgr5 krb5kdc[25667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050718, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:58.959800 26249 init.cc:377] Logged in from keytab as kudu/127.16.178.3@KRBTEST.COM (short username kudu)
I20260806 21:11:58.960986 26249 webserver.cc:460] Webserver started at http://127.16.178.3:43243/ using document root <none> and password file <none>
I20260806 21:11:58.961702 26249 fs_manager.cc:359] Metadata directory not provided
I20260806 21:11:58.961787 26249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:11:58.962019 26249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:11:58.964462 26249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-2/data/instance:
uuid: "1fe7b416ae95496bb08a58a740150dc2"
format_stamp: "Formatted at 2026-08-06 21:11:58 on dist-test-slave-lgr5"
server_key: "996ef74810d7182cab686eea1993227b"
server_key_iv: "d41ca519b858e93a7b753674dcbffda5"
server_key_version: "encryptionkey@0"
I20260806 21:11:58.965111 26249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance:
uuid: "1fe7b416ae95496bb08a58a740150dc2"
format_stamp: "Formatted at 2026-08-06 21:11:58 on dist-test-slave-lgr5"
server_key: "996ef74810d7182cab686eea1993227b"
server_key_iv: "d41ca519b858e93a7b753674dcbffda5"
server_key_version: "encryptionkey@0"
I20260806 21:11:58.969302 26249 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 21:11:58.972569 26272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:58.974390 26249 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260806 21:11:58.974546 26249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-2/wal
uuid: "1fe7b416ae95496bb08a58a740150dc2"
format_stamp: "Formatted at 2026-08-06 21:11:58 on dist-test-slave-lgr5"
server_key: "996ef74810d7182cab686eea1993227b"
server_key_iv: "d41ca519b858e93a7b753674dcbffda5"
server_key_version: "encryptionkey@0"
I20260806 21:11:58.974766 26249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:11:58.993484 26249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:11:58.998517 26249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:11:58.998744 26249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:11:58.999430 26249 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:11:59.000629 26249 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:11:59.000698 26249 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:59.000764 26249 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:11:59.000808 26249 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:11:59.012459 26249 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.3:40761
I20260806 21:11:59.012503 26447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.3:40761 every 8 connection(s)
I20260806 21:11:59.013466 26249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
I20260806 21:11:59.015272 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 26249
I20260806 21:11:59.015405 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance
I20260806 21:11:59.015719 17096 external_mini_cluster.cc:1428] Setting key b344dd623afd3206814244c033b90851
Aug 06 21:11:59 dist-test-slave-lgr5 krb5kdc[25667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050718, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.3@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:11:59.028946 25803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:59.015470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:37593 (local address 127.16.178.62:34843)
0806 21:11:59.015636 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:59.015641 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:59.016611 (+ 970us) server_negotiation.cc:407] Connection header received
0806 21:11:59.018345 (+ 1734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:59.018348 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:59.018408 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:59.018480 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:59.019773 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:59.020401 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:59.021711 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:59.021937 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:59.022030 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:59.024870 (+ 2840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:59.024874 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:59.024875 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:59.024908 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:59.026419 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:59.027048 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:59.027052 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:59.027053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:59.027188 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:59.027574 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:59.027577 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:59.027579 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:59.027793 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:59.027883 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:59.028643 (+ 760us) server_negotiation.cc:299] Negotiation successful
0806 21:11:59.028798 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 21:11:59.029904 26451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:59.015874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:34843 (local address 127.16.178.3:37593)
0806 21:11:59.016454 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0806 21:11:59.016516 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 21:11:59.018127 (+ 1611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:11:59.018683 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:11:59.018690 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:11:59.019085 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:11:59.019618 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:59.019634 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:59.020531 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:59.020535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:59.021572 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:11:59.021584 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:11:59.022098 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:11:59.022102 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:11:59.022296 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:59.022901 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:11:59.022920 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:11:59.024620 (+ 1700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:11:59.026576 (+ 1956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:59.026584 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:59.026601 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:59.026922 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:59.027289 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:11:59.027292 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:11:59.027293 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:11:59.027455 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:11:59.027919 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:11:59.027925 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:11:59.028343 (+ 418us) client_negotiation.cc:769] Sending connection context
0806 21:11:59.028546 (+ 203us) client_negotiation.cc:240] Negotiation successful
0806 21:11:59.028925 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":179,"threads_started":1}
I20260806 21:11:59.030959 26449 heartbeater.cc:347] Connected to a master server at 127.16.178.62:34843
I20260806 21:11:59.031220 26449 heartbeater.cc:464] Registering TS with master...
I20260806 21:11:59.031752 26449 heartbeater.cc:511] Master 127.16.178.62:34843 requested a full tablet report, sending...
I20260806 21:11:59.032871 25743 ts_manager.cc:194] Registered new tserver with Master: 1fe7b416ae95496bb08a58a740150dc2 (127.16.178.3:40761)
I20260806 21:11:59.033432 25743 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.3@KRBTEST.COM'} at 127.16.178.3:37593
I20260806 21:11:59.042151 17096 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/krb5kdc/test-admin.keytab.
Aug 06 21:11:59 dist-test-slave-lgr5 krb5kdc[25667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:59.060926 17096 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260806 21:11:59.063910 26459 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:11:59.062486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50028 (local address 127.16.178.1:40733)
0806 21:11:59.062802 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:59.062819 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:59.062846 (+ 27us) server_negotiation.cc:407] Connection header received
0806 21:11:59.062905 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:59.062910 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:59.063105 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:59.063234 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:59.063734 (+ 500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50028: BlockingRecv error: recv got EOF from 127.0.0.1:50028 (error 108)
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":98,"threads_started":1}
W20260806 21:11:59.064132 17096 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/krb5kdc/krb5cc' of type FILE
Aug 06 21:11:59 dist-test-slave-lgr5 krb5kdc[25667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:59.065306 17096 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 21:11:59 dist-test-slave-lgr5 krb5kdc[25667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:59.066290 17096 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 21:11:59 dist-test-slave-lgr5 krb5kdc[25667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.178.1@KRBTEST.COM
I20260806 21:11:59.077641 26459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:59.067411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50032 (local address 127.16.178.1:40733)
0806 21:11:59.067674 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:59.067678 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:59.067692 (+ 14us) server_negotiation.cc:407] Connection header received
0806 21:11:59.067738 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:59.067741 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:59.067783 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:59.067854 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:59.068465 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:59.069270 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:59.070163 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:59.070630 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:59.070726 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:59.073372 (+ 2646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:59.073386 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:59.073426 (+ 40us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:59.073459 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:59.075322 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:59.075878 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:59.075884 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:59.075885 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:59.075974 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:59.076309 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:59.076311 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:59.076312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:59.076669 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:59.076857 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:59.077228 (+ 371us) server_negotiation.cc:299] Negotiation successful
0806 21:11:59.077494 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186}
W20260806 21:11:59.082841 26459 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:11:59.081564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50038 (local address 127.16.178.1:40733)
0806 21:11:59.081708 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:59.081712 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:59.081729 (+ 17us) server_negotiation.cc:407] Connection header received
0806 21:11:59.081917 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:59.081919 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:59.081967 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:59.082038 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:59.082707 (+ 669us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50038: BlockingRecv error: recv got EOF from 127.0.0.1:50038 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260806 21:11:59.082953 17096 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/krb5kdc/krb5cc' of type FILE
Aug 06 21:11:59 dist-test-slave-lgr5 krb5kdc[25667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:59.084128 17096 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 21:11:59 dist-test-slave-lgr5 krb5kdc[25667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:59.084996 17096 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 21:11:59 dist-test-slave-lgr5 krb5kdc[25667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.178.1@KRBTEST.COM
I20260806 21:11:59.098326 26459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:59.085984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50044 (local address 127.16.178.1:40733)
0806 21:11:59.086127 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:59.086132 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:59.086215 (+ 83us) server_negotiation.cc:407] Connection header received
0806 21:11:59.086366 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:59.086369 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:59.086414 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:59.086487 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:59.087063 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:59.087660 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:59.088614 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:59.088848 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:59.088937 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:59.095183 (+ 6246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:59.095188 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:59.095190 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:59.095232 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:59.096611 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:59.097113 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:59.097116 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:59.097117 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:59.097198 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:59.097468 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:59.097470 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:59.097471 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:59.097656 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:59.097754 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:59.098053 (+ 299us) server_negotiation.cc:299] Negotiation successful
0806 21:11:59.098191 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260806 21:11:59.101295 26459 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:11:59.100369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50046 (local address 127.16.178.1:40733)
0806 21:11:59.100485 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:59.100488 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:59.100629 (+ 141us) server_negotiation.cc:407] Connection header received
0806 21:11:59.100742 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:59.100745 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:59.100783 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:59.100849 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:59.101191 (+ 342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50046: BlockingRecv error: recv got EOF from 127.0.0.1:50046 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260806 21:11:59.101652 17096 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786050686797304-17096-0/krb5kdc/krb5cc' of type FILE
Aug 06 21:11:59 dist-test-slave-lgr5 krb5kdc[25667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:59.102769 17096 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 21:11:59 dist-test-slave-lgr5 krb5kdc[25667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:11:59.103751 17096 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 21:11:59 dist-test-slave-lgr5 krb5kdc[25667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.178.1@KRBTEST.COM
I20260806 21:11:59.112411 26459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:11:59.104705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50060 (local address 127.16.178.1:40733)
0806 21:11:59.104855 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 21:11:59.104859 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:11:59.104974 (+ 115us) server_negotiation.cc:407] Connection header received
0806 21:11:59.105072 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:11:59.105076 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:11:59.105116 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:11:59.105176 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:11:59.105719 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:59.106307 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:59.107121 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:11:59.107328 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:11:59.107414 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:11:59.109547 (+ 2133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:11:59.109552 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:11:59.109554 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:11:59.109578 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:11:59.110705 (+ 1127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:59.111090 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:59.111094 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:59.111096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:59.111171 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:11:59.111570 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:11:59.111574 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:11:59.111576 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:11:59.111753 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:11:59.111897 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 21:11:59.112144 (+ 247us) server_negotiation.cc:299] Negotiation successful
0806 21:11:59.112258 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 21:11:59.113552 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 25836
I20260806 21:11:59.127151 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 26046
I20260806 21:11:59.158442 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 26249
I20260806 21:11:59.169255 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 25693
2026-08-06T21:11:59Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2384 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:11:59 dist-test-slave-lgr5 krb5kdc[26480](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:11:59 dist-test-slave-lgr5 krb5kdc[26480](info): set up 2 sockets
Aug 06 21:11:59 dist-test-slave-lgr5 krb5kdc[26480](info): commencing operation
krb5kdc: starting...
W20260806 21:12:00.350267 17096 mini_kdc.cc:121] Time spent starting KDC: real 1.148s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Aug 06 21:12:00 dist-test-slave-lgr5 krb5kdc[26480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:12:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:12:00Z Disabled control of system clock
WARNING: no policy specified for oryx/127.16.178.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.178.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.178.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.62.keytab.
Entry for principal oryx/127.16.178.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.62.keytab.
I20260806 21:12:00.484315 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:44091
--webserver_interface=127.16.178.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.62.keytab
--principal=oryx/127.16.178.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:43991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:44091
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:12:00.600551 26494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:12:00.600857 26494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:12:00.600926 26494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:12:00.604995 26494 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:12:00.605073 26494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:12:00.605101 26494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:12:00.605121 26494 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:12:00.605142 26494 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:12:00.610776 26494 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:43991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:44091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.62.keytab
--principal=oryx/127.16.178.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:44091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.178.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26494
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:12:00.612052 26494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:12:00.613196 26494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:12:00.621985 26504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:00.621973 26509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:00.621971 26503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:00.622346 26494 server_base.cc:1034] running on GCE node
I20260806 21:12:00.623031 26494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:12:00.624380 26494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:12:00.625561 26494 hybrid_clock.cc:648] HybridClock initialized: now 1786050720625545 us; error 39 us; skew 500 ppm
Aug 06 21:12:00 dist-test-slave-lgr5 krb5kdc[26480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050720, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.178.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:12:00.628444 26494 init.cc:377] Logged in from keytab as oryx/127.16.178.62@KRBTEST.COM (short username oryx)
I20260806 21:12:00.629388 26494 webserver.cc:460] Webserver started at http://127.16.178.62:37483/ using document root <none> and password file <none>
I20260806 21:12:00.629930 26494 fs_manager.cc:359] Metadata directory not provided
I20260806 21:12:00.629989 26494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:12:00.630165 26494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:12:00.632320 26494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "b90b03b56af14eeea8b3a169d902cf3e"
format_stamp: "Formatted at 2026-08-06 21:12:00 on dist-test-slave-lgr5"
server_key: "85ba86b853afbe89d593f4ef45c683bc"
server_key_iv: "dcc11dead2b78217f4b15de53347c24c"
server_key_version: "encryptionkey@0"
I20260806 21:12:00.632862 26494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "b90b03b56af14eeea8b3a169d902cf3e"
format_stamp: "Formatted at 2026-08-06 21:12:00 on dist-test-slave-lgr5"
server_key: "85ba86b853afbe89d593f4ef45c683bc"
server_key_iv: "dcc11dead2b78217f4b15de53347c24c"
server_key_version: "encryptionkey@0"
I20260806 21:12:00.636641 26494 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 21:12:00.639164 26522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:00.640578 26494 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:12:00.640698 26494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "b90b03b56af14eeea8b3a169d902cf3e"
format_stamp: "Formatted at 2026-08-06 21:12:00 on dist-test-slave-lgr5"
server_key: "85ba86b853afbe89d593f4ef45c683bc"
server_key_iv: "dcc11dead2b78217f4b15de53347c24c"
server_key_version: "encryptionkey@0"
I20260806 21:12:00.640816 26494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:12:00.659430 26494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:12:00.662432 26494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:12:00.662637 26494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:12:00.670044 26494 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.62:44091
I20260806 21:12:00.670341 26598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.62:44091 every 8 connection(s)
I20260806 21:12:00.671222 26494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:12:00.671654 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 26494
I20260806 21:12:00.671789 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:12:00.672174 17096 external_mini_cluster.cc:1428] Setting key af90ac92798594a3ffb9dec56feca996
I20260806 21:12:00.674798 26599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:12:00.682130 26599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e: Bootstrap starting.
Aug 06 21:12:00 dist-test-slave-lgr5 krb5kdc[26480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM
I20260806 21:12:00.684741 26599 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e: Neither blocks nor log segments found. Creating new log.
I20260806 21:12:00.685647 26599 log.cc:824] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e: Log is configured to *not* fsync() on all Append() calls
I20260806 21:12:00.687750 26605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:00.673687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58816 (local address 127.16.178.62:44091)
0806 21:12:00.674160 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:00.674183 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:00.674227 (+ 44us) server_negotiation.cc:407] Connection header received
0806 21:12:00.674938 (+ 711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:00.674963 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:00.675357 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:00.675661 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:00.676678 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:00.678117 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:00.679306 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:00.679839 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:00.679982 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:00.683013 (+ 3031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:00.683019 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:00.683034 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:00.683081 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:00.684796 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:00.685289 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:00.685294 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:00.685299 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:00.685451 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:00.685746 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:00.685750 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:00.685752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:00.686254 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:00.686507 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:00.686691 (+ 184us) server_negotiation.cc:299] Negotiation successful
0806 21:12:00.686989 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":136,"threads_started":1}
I20260806 21:12:00.687857 26599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e: No bootstrap required, opened a new log
I20260806 21:12:00.690930 26599 raft_consensus.cc:348] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } }
I20260806 21:12:00.691172 26599 raft_consensus.cc:374] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:12:00.691224 26599 raft_consensus.cc:729] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b90b03b56af14eeea8b3a169d902cf3e, State: Initialized, Role: FOLLOWER
I20260806 21:12:00.691625 26599 consensus_queue.cc:260] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } }
I20260806 21:12:00.691749 26599 raft_consensus.cc:388] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:12:00.691811 26599 raft_consensus.cc:482] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:12:00.691907 26599 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:12:00.692864 26599 raft_consensus.cc:504] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } }
I20260806 21:12:00.693190 26599 leader_election.cc:302] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b90b03b56af14eeea8b3a169d902cf3e; no voters:
I20260806 21:12:00.693480 26599 leader_election.cc:288] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:12:00.693624 26607 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:12:00.693846 26607 raft_consensus.cc:686] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e [term 1 LEADER]: Becoming Leader. State: Replica: b90b03b56af14eeea8b3a169d902cf3e, State: Running, Role: LEADER
I20260806 21:12:00.694236 26607 consensus_queue.cc:237] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } }
I20260806 21:12:00.694802 26599 sys_catalog.cc:564] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:12:00.695937 26609 sys_catalog.cc:455] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e [sys.catalog]: SysCatalogTable state changed. Reason: New leader b90b03b56af14eeea8b3a169d902cf3e. Latest consensus state: current_term: 1 leader_uuid: "b90b03b56af14eeea8b3a169d902cf3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } }
I20260806 21:12:00.696069 26609 sys_catalog.cc:458] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e [sys.catalog]: This master's current role is: LEADER
I20260806 21:12:00.696475 26618 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:12:00.696440 26608 sys_catalog.cc:455] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b90b03b56af14eeea8b3a169d902cf3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } }
I20260806 21:12:00.697237 26608 sys_catalog.cc:458] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e [sys.catalog]: This master's current role is: LEADER
I20260806 21:12:00.699368 26618 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:12:00.705070 26618 catalog_manager.cc:1347] Generated new cluster ID: 3fb66fa414d5482586f34a12c979adfa
I20260806 21:12:00.705150 26618 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:12:00.726394 26618 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:12:00.727705 26618 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:12:00.732933 26618 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e: Generated new TSK 0
I20260806 21:12:00.733789 26618 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.16.178.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.178.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.178.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.1.keytab.
Entry for principal oryx/127.16.178.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.1.keytab.
I20260806 21:12:00.763181 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.1:0
--local_ip_for_outbound_sockets=127.16.178.1
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:44091
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.1.keytab
--principal=oryx/127.16.178.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:43991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:12:00.879727 26633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:12:00.879999 26633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:12:00.880043 26633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:12:00.884030 26633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:12:00.884119 26633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:12:00.884196 26633 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.1
I20260806 21:12:00.889447 26633 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:43991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.1.keytab
--principal=oryx/127.16.178.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:44091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.1
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:12:00.890491 26633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:12:00.891479 26633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:12:00.901758 26648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:00.901759 26641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:00.901857 26633 server_base.cc:1034] running on GCE node
W20260806 21:12:00.901799 26645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:00.902444 26633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:12:00.903129 26633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:12:00.904290 26633 hybrid_clock.cc:648] HybridClock initialized: now 1786050720904273 us; error 30 us; skew 500 ppm
Aug 06 21:12:00 dist-test-slave-lgr5 krb5kdc[26480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050720, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.178.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:12:00.907147 26633 init.cc:377] Logged in from keytab as oryx/127.16.178.1@KRBTEST.COM (short username oryx)
I20260806 21:12:00.908129 26633 webserver.cc:460] Webserver started at http://127.16.178.1:35755/ using document root <none> and password file <none>
I20260806 21:12:00.908708 26633 fs_manager.cc:359] Metadata directory not provided
I20260806 21:12:00.908779 26633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:12:00.908987 26633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:12:00.911147 26633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-0/data/instance:
uuid: "97b7760ee07347c2990d40a6cbba0360"
format_stamp: "Formatted at 2026-08-06 21:12:00 on dist-test-slave-lgr5"
server_key: "fd42abb9789eb3d6c3be88ca1ed0e41a"
server_key_iv: "3ea1cb8bb312f5fd68814fd77beada08"
server_key_version: "encryptionkey@0"
I20260806 21:12:00.911724 26633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance:
uuid: "97b7760ee07347c2990d40a6cbba0360"
format_stamp: "Formatted at 2026-08-06 21:12:00 on dist-test-slave-lgr5"
server_key: "fd42abb9789eb3d6c3be88ca1ed0e41a"
server_key_iv: "3ea1cb8bb312f5fd68814fd77beada08"
server_key_version: "encryptionkey@0"
I20260806 21:12:00.915807 26633 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260806 21:12:00.918423 26658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:00.919781 26633 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 21:12:00.919919 26633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-0/wal
uuid: "97b7760ee07347c2990d40a6cbba0360"
format_stamp: "Formatted at 2026-08-06 21:12:00 on dist-test-slave-lgr5"
server_key: "fd42abb9789eb3d6c3be88ca1ed0e41a"
server_key_iv: "3ea1cb8bb312f5fd68814fd77beada08"
server_key_version: "encryptionkey@0"
I20260806 21:12:00.920047 26633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:12:00.931617 26633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:12:00.935258 26633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:12:00.935436 26633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:12:00.936070 26633 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:12:00.937134 26633 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:12:00.937196 26633 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:00.937245 26633 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:12:00.937259 26633 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:00.947139 26633 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.1:46257
I20260806 21:12:00.947180 26830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.1:46257 every 8 connection(s)
I20260806 21:12:00.948375 26633 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
I20260806 21:12:00.950752 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 26633
I20260806 21:12:00.950906 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance
I20260806 21:12:00.951305 17096 external_mini_cluster.cc:1428] Setting key d768819352b499fce994a2e034face30
Aug 06 21:12:00 dist-test-slave-lgr5 krb5kdc[26480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050720, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.178.1@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM
WARNING: no policy specified for oryx/127.16.178.2@KRBTEST.COM; defaulting to no policy
I20260806 21:12:00.964475 26605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:00.950560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:56653 (local address 127.16.178.62:44091)
0806 21:12:00.950720 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:00.950723 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:00.951895 (+ 1172us) server_negotiation.cc:407] Connection header received
0806 21:12:00.953438 (+ 1543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:00.953441 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:00.953506 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:00.953575 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:00.954868 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:00.955483 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:00.957217 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:00.957486 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:00.957583 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:00.960749 (+ 3166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:00.960752 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:00.960754 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:00.960792 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:00.962330 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:00.962845 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:00.962848 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:00.962849 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:00.962927 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:00.963280 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:00.963283 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:00.963284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:00.963517 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:00.963596 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:00.964167 (+ 571us) server_negotiation.cc:299] Negotiation successful
0806 21:12:00.964312 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.178.2@KRBTEST.COM" created.
I20260806 21:12:00.965354 26835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:00.951087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44091 (local address 127.16.178.1:56653)
0806 21:12:00.951691 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:00.951752 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:00.953222 (+ 1470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:00.953772 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:00.953779 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:00.954174 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:00.954689 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:00.954704 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:00.955629 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:00.955633 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:00.957056 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:00.957069 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:00.957665 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:00.957670 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:00.957903 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:00.958603 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:00.958626 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:00.960525 (+ 1899us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:12:00.962453 (+ 1928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:00.962461 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:00.962481 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:00.962739 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:00.963001 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:00.963003 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:00.963005 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:00.963170 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:00.963604 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:12:00.963610 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:12:00.963878 (+ 268us) client_negotiation.cc:769] Sending connection context
0806 21:12:00.964057 (+ 179us) client_negotiation.cc:240] Negotiation successful
0806 21:12:00.964413 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":172,"threads_started":1}
I20260806 21:12:00.966641 26831 heartbeater.cc:347] Connected to a master server at 127.16.178.62:44091
I20260806 21:12:00.966887 26831 heartbeater.cc:464] Registering TS with master...
I20260806 21:12:00.967407 26831 heartbeater.cc:511] Master 127.16.178.62:44091 requested a full tablet report, sending...
I20260806 21:12:00.969053 26545 ts_manager.cc:194] Registered new tserver with Master: 97b7760ee07347c2990d40a6cbba0360 (127.16.178.1:46257)
I20260806 21:12:00.970309 26545 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.178.1@KRBTEST.COM'} at 127.16.178.1:56653
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.178.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.2.keytab.
Entry for principal oryx/127.16.178.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.2.keytab.
I20260806 21:12:00.981454 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.2:0
--local_ip_for_outbound_sockets=127.16.178.2
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:44091
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.2.keytab
--principal=oryx/127.16.178.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:43991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:12:01.096966 26841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:12:01.097255 26841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:12:01.097316 26841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:12:01.101604 26841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:12:01.101689 26841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:12:01.101776 26841 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.2
I20260806 21:12:01.107458 26841 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:43991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.2.keytab
--principal=oryx/127.16.178.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:44091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.2
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:12:01.108716 26841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:12:01.109783 26841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:12:01.120724 26848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:01.120739 26849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:01.120736 26853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:01.121167 26841 server_base.cc:1034] running on GCE node
I20260806 21:12:01.121697 26841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:12:01.122419 26841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:12:01.123553 26841 hybrid_clock.cc:648] HybridClock initialized: now 1786050721123531 us; error 36 us; skew 500 ppm
Aug 06 21:12:01 dist-test-slave-lgr5 krb5kdc[26480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050721, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.178.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:12:01.126647 26841 init.cc:377] Logged in from keytab as oryx/127.16.178.2@KRBTEST.COM (short username oryx)
I20260806 21:12:01.127692 26841 webserver.cc:460] Webserver started at http://127.16.178.2:38187/ using document root <none> and password file <none>
I20260806 21:12:01.128391 26841 fs_manager.cc:359] Metadata directory not provided
I20260806 21:12:01.128474 26841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:12:01.128727 26841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:12:01.131078 26841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-1/data/instance:
uuid: "e8a457dc20ec498da1b35c91b59c632a"
format_stamp: "Formatted at 2026-08-06 21:12:01 on dist-test-slave-lgr5"
server_key: "b6a64ee95845b2755274528745563f62"
server_key_iv: "5498fdcd56928a00942870a8ab7090d5"
server_key_version: "encryptionkey@0"
I20260806 21:12:01.131701 26841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance:
uuid: "e8a457dc20ec498da1b35c91b59c632a"
format_stamp: "Formatted at 2026-08-06 21:12:01 on dist-test-slave-lgr5"
server_key: "b6a64ee95845b2755274528745563f62"
server_key_iv: "5498fdcd56928a00942870a8ab7090d5"
server_key_version: "encryptionkey@0"
I20260806 21:12:01.135876 26841 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 21:12:01.138783 26864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:01.140316 26841 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:12:01.140450 26841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-1/wal
uuid: "e8a457dc20ec498da1b35c91b59c632a"
format_stamp: "Formatted at 2026-08-06 21:12:01 on dist-test-slave-lgr5"
server_key: "b6a64ee95845b2755274528745563f62"
server_key_iv: "5498fdcd56928a00942870a8ab7090d5"
server_key_version: "encryptionkey@0"
I20260806 21:12:01.140569 26841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:12:01.170120 26841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:12:01.173635 26841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:12:01.173848 26841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:12:01.174492 26841 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:12:01.175431 26841 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:12:01.175487 26841 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:01.175549 26841 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:12:01.175578 26841 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:01.185134 26841 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.2:41469
I20260806 21:12:01.185207 27040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.2:41469 every 8 connection(s)
I20260806 21:12:01.186152 26841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
I20260806 21:12:01.189810 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 26841
I20260806 21:12:01.189944 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance
I20260806 21:12:01.190296 17096 external_mini_cluster.cc:1428] Setting key 9c8c64c3726f985f785e78ad6f7c1548
Aug 06 21:12:01 dist-test-slave-lgr5 krb5kdc[26480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050721, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.178.2@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM
WARNING: no policy specified for oryx/127.16.178.3@KRBTEST.COM; defaulting to no policy
I20260806 21:12:01.201674 26605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:01.188244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:57689 (local address 127.16.178.62:44091)
0806 21:12:01.188410 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:01.188415 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:01.189371 (+ 956us) server_negotiation.cc:407] Connection header received
0806 21:12:01.191057 (+ 1686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:01.191060 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:01.191119 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:01.191206 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:01.192733 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:01.193281 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:01.194744 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:01.194972 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:01.195107 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:01.198008 (+ 2901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:01.198011 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:01.198014 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:01.198042 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:01.199402 (+ 1360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:01.199949 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:01.199952 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:01.199953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:01.200038 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:01.200404 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:01.200407 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:01.200408 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:01.200620 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:01.200713 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:01.201325 (+ 612us) server_negotiation.cc:299] Negotiation successful
0806 21:12:01.201476 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 21:12:01.202647 27046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:01.188723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44091 (local address 127.16.178.2:57689)
0806 21:12:01.189205 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:01.189252 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:01.190814 (+ 1562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:01.191366 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:01.191373 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:01.191822 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:01.192549 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:01.192571 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:01.193425 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:01.193428 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:01.194578 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:01.194590 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:01.195278 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:01.195283 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:01.195480 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:01.196210 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:01.196228 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:01.197810 (+ 1582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:12:01.199534 (+ 1724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:01.199541 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:01.199561 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:01.199837 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:01.200138 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:01.200142 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:01.200144 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:01.200300 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:01.200727 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:12:01.200736 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:12:01.201041 (+ 305us) client_negotiation.cc:769] Sending connection context
0806 21:12:01.201249 (+ 208us) client_negotiation.cc:240] Negotiation successful
0806 21:12:01.201641 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":130,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.178.3@KRBTEST.COM" created.
I20260806 21:12:01.204162 27043 heartbeater.cc:347] Connected to a master server at 127.16.178.62:44091
I20260806 21:12:01.204443 27043 heartbeater.cc:464] Registering TS with master...
I20260806 21:12:01.205045 27043 heartbeater.cc:511] Master 127.16.178.62:44091 requested a full tablet report, sending...
I20260806 21:12:01.206444 26545 ts_manager.cc:194] Registered new tserver with Master: e8a457dc20ec498da1b35c91b59c632a (127.16.178.2:41469)
I20260806 21:12:01.207098 26545 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.178.2@KRBTEST.COM'} at 127.16.178.2:57689
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.178.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.3.keytab.
Entry for principal oryx/127.16.178.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.3.keytab.
I20260806 21:12:01.222898 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.3:0
--local_ip_for_outbound_sockets=127.16.178.3
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:44091
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.3.keytab
--principal=oryx/127.16.178.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:43991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:12:01.342825 27051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:12:01.343142 27051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:12:01.343186 27051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:12:01.347679 27051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:12:01.347764 27051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:12:01.347828 27051 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.3
I20260806 21:12:01.353811 27051 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:43991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.3.keytab
--principal=oryx/127.16.178.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:44091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.3
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:12:01.355146 27051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:12:01.356403 27051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:12:01.367588 27064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:01.367769 27051 server_base.cc:1034] running on GCE node
W20260806 21:12:01.367578 27060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:01.367578 27058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:01.368418 27051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:12:01.369180 27051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:12:01.370349 27051 hybrid_clock.cc:648] HybridClock initialized: now 1786050721370337 us; error 34 us; skew 500 ppm
Aug 06 21:12:01 dist-test-slave-lgr5 krb5kdc[26480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050721, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.178.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:12:01.373301 27051 init.cc:377] Logged in from keytab as oryx/127.16.178.3@KRBTEST.COM (short username oryx)
I20260806 21:12:01.374290 27051 webserver.cc:460] Webserver started at http://127.16.178.3:33583/ using document root <none> and password file <none>
I20260806 21:12:01.374882 27051 fs_manager.cc:359] Metadata directory not provided
I20260806 21:12:01.374955 27051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:12:01.375200 27051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:12:01.377475 27051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-2/data/instance:
uuid: "afab74cbf9344028ad587dc2c7896c5d"
format_stamp: "Formatted at 2026-08-06 21:12:01 on dist-test-slave-lgr5"
server_key: "338fb855bd9c62c1f8b849130efd01df"
server_key_iv: "25bcf9a02583fc9dd5a02c6166d49751"
server_key_version: "encryptionkey@0"
I20260806 21:12:01.378156 27051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance:
uuid: "afab74cbf9344028ad587dc2c7896c5d"
format_stamp: "Formatted at 2026-08-06 21:12:01 on dist-test-slave-lgr5"
server_key: "338fb855bd9c62c1f8b849130efd01df"
server_key_iv: "25bcf9a02583fc9dd5a02c6166d49751"
server_key_version: "encryptionkey@0"
I20260806 21:12:01.382443 27051 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:12:01.385097 27074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:01.386346 27051 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 21:12:01.386495 27051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-2/wal
uuid: "afab74cbf9344028ad587dc2c7896c5d"
format_stamp: "Formatted at 2026-08-06 21:12:01 on dist-test-slave-lgr5"
server_key: "338fb855bd9c62c1f8b849130efd01df"
server_key_iv: "25bcf9a02583fc9dd5a02c6166d49751"
server_key_version: "encryptionkey@0"
I20260806 21:12:01.386607 27051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:12:01.420521 27051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:12:01.424398 27051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:12:01.424633 27051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:12:01.425331 27051 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:12:01.426290 27051 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:12:01.426340 27051 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:01.426386 27051 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:12:01.426419 27051 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:01.435746 27051 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.3:44489
I20260806 21:12:01.435803 27254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.3:44489 every 8 connection(s)
I20260806 21:12:01.436849 27051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
I20260806 21:12:01.441568 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 27051
I20260806 21:12:01.441720 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance
I20260806 21:12:01.442041 17096 external_mini_cluster.cc:1428] Setting key 19a5927f97b648ebd292633924d72bf5
Aug 06 21:12:01 dist-test-slave-lgr5 krb5kdc[26480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050721, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.178.3@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM
I20260806 21:12:01.452939 26605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:01.439153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:49573 (local address 127.16.178.62:44091)
0806 21:12:01.439328 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:01.439333 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:01.440458 (+ 1125us) server_negotiation.cc:407] Connection header received
0806 21:12:01.442193 (+ 1735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:01.442197 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:01.442266 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:01.442350 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:01.444063 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:01.444833 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:01.446296 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:01.446548 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:01.446647 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:01.449303 (+ 2656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:01.449306 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:01.449308 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:01.449338 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:01.450731 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:01.451309 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:01.451313 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:01.451314 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:01.451417 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:01.451731 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:01.451733 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:01.451735 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:01.451949 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:01.452050 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:01.452673 (+ 623us) server_negotiation.cc:299] Negotiation successful
0806 21:12:01.452804 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 21:12:01.453802 27258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:01.439628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44091 (local address 127.16.178.3:49573)
0806 21:12:01.440277 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:01.440337 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:01.441842 (+ 1505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:01.442550 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:01.442558 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:01.443082 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:01.443857 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:01.443877 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:01.445002 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:01.445005 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:01.446155 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:01.446169 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:01.446696 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:01.446698 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:01.446859 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:01.447493 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:01.447512 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:01.449110 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:12:01.450837 (+ 1727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:01.450843 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:01.450857 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:01.451143 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:01.451509 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:01.451512 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:01.451513 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:01.451657 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:01.452048 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:12:01.452055 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:12:01.452386 (+ 331us) client_negotiation.cc:769] Sending connection context
0806 21:12:01.452567 (+ 181us) client_negotiation.cc:240] Negotiation successful
0806 21:12:01.452866 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":144,"threads_started":1}
I20260806 21:12:01.455075 27255 heartbeater.cc:347] Connected to a master server at 127.16.178.62:44091
I20260806 21:12:01.455327 27255 heartbeater.cc:464] Registering TS with master...
I20260806 21:12:01.455894 27255 heartbeater.cc:511] Master 127.16.178.62:44091 requested a full tablet report, sending...
I20260806 21:12:01.457055 26545 ts_manager.cc:194] Registered new tserver with Master: afab74cbf9344028ad587dc2c7896c5d (127.16.178.3:44489)
I20260806 21:12:01.457628 26545 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.178.3@KRBTEST.COM'} at 127.16.178.3:49573
I20260806 21:12:01.469120 17096 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.16.178.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.178.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.178.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.61.keytab.
Entry for principal oryx/127.16.178.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.61.keytab.
I20260806 21:12:01.497579 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.61:43151
--webserver_interface=127.16.178.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.61.keytab
--principal=oryx/127.16.178.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:43991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:44091,127.16.178.61:43151
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:12:01.611788 27263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:12:01.612171 27263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:12:01.612239 27263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:12:01.616387 27263 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:12:01.616460 27263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:12:01.616480 27263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:12:01.616496 27263 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:12:01.616510 27263 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:12:01.622251 27263 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:43991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:44091,127.16.178.61:43151
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.61.keytab
--principal=oryx/127.16.178.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.61:43151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.16.178.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27263
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
W20260806 21:12:01.622511 27263 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.16.178.62:44091,127.16.178.61:43151'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260806 21:12:01.623457 27263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:12:01.624610 27263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:12:01.633633 27274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:01.633628 27271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:01.633708 27263 server_base.cc:1034] running on GCE node
W20260806 21:12:01.633628 27272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:01.634408 27263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:12:01.635639 27263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:12:01.636829 27263 hybrid_clock.cc:648] HybridClock initialized: now 1786050721636810 us; error 35 us; skew 500 ppm
Aug 06 21:12:01 dist-test-slave-lgr5 krb5kdc[26480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050721, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.178.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:12:01.639721 27263 init.cc:377] Logged in from keytab as oryx/127.16.178.61@KRBTEST.COM (short username oryx)
I20260806 21:12:01.640652 27263 webserver.cc:460] Webserver started at http://127.16.178.61:44419/ using document root <none> and password file <none>
I20260806 21:12:01.641175 27263 fs_manager.cc:359] Metadata directory not provided
I20260806 21:12:01.641234 27263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:12:01.641419 27263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:12:01.643426 27263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-1/data/instance:
uuid: "66d06f2f26ad497aab505e20b1b54dfd"
format_stamp: "Formatted at 2026-08-06 21:12:01 on dist-test-slave-lgr5"
server_key: "67faf9da9b1dc5013dfa799333000263"
server_key_iv: "6f4c9ccfe21854682e385287470270c5"
server_key_version: "encryptionkey@0"
I20260806 21:12:01.644029 27263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-1/wal/instance:
uuid: "66d06f2f26ad497aab505e20b1b54dfd"
format_stamp: "Formatted at 2026-08-06 21:12:01 on dist-test-slave-lgr5"
server_key: "67faf9da9b1dc5013dfa799333000263"
server_key_iv: "6f4c9ccfe21854682e385287470270c5"
server_key_version: "encryptionkey@0"
I20260806 21:12:01.647852 27263 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 21:12:01.650271 27285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:01.651458 27263 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 21:12:01.651578 27263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-1/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-1/wal
uuid: "66d06f2f26ad497aab505e20b1b54dfd"
format_stamp: "Formatted at 2026-08-06 21:12:01 on dist-test-slave-lgr5"
server_key: "67faf9da9b1dc5013dfa799333000263"
server_key_iv: "6f4c9ccfe21854682e385287470270c5"
server_key_version: "encryptionkey@0"
I20260806 21:12:01.651684 27263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:12:01.687889 27263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:12:01.691315 27263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:12:01.691498 27263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:12:01.698841 27263 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.61:43151
I20260806 21:12:01.699155 27360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.61:43151 every 8 connection(s)
I20260806 21:12:01.699842 27263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-1/data/info.pb
I20260806 21:12:01.702715 27362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:12:01.705104 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 27263
I20260806 21:12:01.706800 27362 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 21:12:01 dist-test-slave-lgr5 krb5kdc[26480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050721, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.178.61@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM
Aug 06 21:12:01 dist-test-slave-lgr5 krb5kdc[26480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.178.61@KRBTEST.COM
I20260806 21:12:01.718824 26605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:01.708530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58818 (local address 127.16.178.62:44091)
0806 21:12:01.708658 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:01.708662 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:01.709045 (+ 383us) server_negotiation.cc:407] Connection header received
0806 21:12:01.709198 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:01.709201 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:01.709248 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:01.709337 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:01.710035 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:01.710784 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:01.712143 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:01.712453 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:01.712553 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:01.714986 (+ 2433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:01.714990 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:01.714992 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:01.715016 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:01.716504 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:01.717060 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:01.717064 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:01.717066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:01.717158 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:01.717481 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:01.717484 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:01.717486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:01.717794 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:01.717890 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:01.718514 (+ 624us) server_negotiation.cc:299] Negotiation successful
0806 21:12:01.718651 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 21:12:01.719862 27369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:01.708433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44091 (local address 127.0.0.1:58818)
0806 21:12:01.708906 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:01.708952 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:01.709073 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:01.709405 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:01.709410 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:01.709720 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:01.709885 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:01.709895 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:01.710912 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:01.710915 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:01.712016 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:01.712026 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:01.712607 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:01.712611 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:01.712709 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:01.713090 (+ 381us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:01.713110 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:01.714784 (+ 1674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:12:01.716609 (+ 1825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:01.716617 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:01.716626 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:01.716955 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:01.717245 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:01.717247 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:01.717249 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:01.717389 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:01.717903 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:12:01.717910 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:12:01.718254 (+ 344us) client_negotiation.cc:769] Sending connection context
0806 21:12:01.718434 (+ 180us) client_negotiation.cc:240] Negotiation successful
0806 21:12:01.718766 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"mutex_wait_us":62,"thread_start_us":148,"threads_started":1}
I20260806 21:12:01.719862 27366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:01.706667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36218 (local address 127.16.178.61:43151)
0806 21:12:01.707084 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:01.707101 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:01.707138 (+ 37us) server_negotiation.cc:407] Connection header received
0806 21:12:01.707798 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:01.707813 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:01.708187 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:01.708499 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:01.709344 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:01.710602 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:01.711723 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:01.712244 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:01.712388 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:01.715365 (+ 2977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:01.715371 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:01.715384 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:01.715411 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:01.717063 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:01.717488 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:01.717494 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:01.717496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:01.717568 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:01.717833 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:01.717835 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:01.717836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:01.718261 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:01.718500 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:01.718722 (+ 222us) server_negotiation.cc:299] Negotiation successful
0806 21:12:01.718842 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":269,"spinlock_wait_cycles":25984,"thread_start_us":109,"threads_started":1}
I20260806 21:12:01.721383 27362 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.178.61" port: 43151 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 21:12:01 dist-test-slave-lgr5 krb5kdc[26480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050721, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.178.61@KRBTEST.COM for oryx/127.16.178.61@KRBTEST.COM
I20260806 21:12:01.728745 27369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:01.721875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.61:43151 (local address 127.0.0.1:36220)
0806 21:12:01.721985 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:01.722000 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:01.722078 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:01.722265 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:01.722267 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:01.722430 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:01.722538 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:01.722543 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:01.723213 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:01.723215 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:01.723771 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:01.723781 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:01.724285 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:01.724289 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:01.724386 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:01.724854 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:01.724866 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:01.726046 (+ 1180us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:12:01.727494 (+ 1448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:01.727496 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:01.727498 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:01.727674 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:01.727909 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:01.727911 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:01.727912 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:01.727976 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:01.728309 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:12:01.728311 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:12:01.728398 (+ 87us) client_negotiation.cc:769] Sending connection context
0806 21:12:01.728489 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 21:12:01.728597 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 21:12:01.728777 27366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:01.721997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36220 (local address 127.16.178.61:43151)
0806 21:12:01.722088 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:01.722092 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:01.722105 (+ 13us) server_negotiation.cc:407] Connection header received
0806 21:12:01.722142 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:01.722144 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:01.722180 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:01.722238 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:01.722621 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:01.723113 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:01.723900 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:01.724165 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:01.724245 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:01.726192 (+ 1947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:01.726195 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:01.726197 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:01.726218 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:01.727397 (+ 1179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:01.727756 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:01.727758 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:01.727759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:01.727833 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:01.728034 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:01.728036 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:01.728037 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:01.728232 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:01.728305 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:01.728535 (+ 230us) server_negotiation.cc:299] Negotiation successful
0806 21:12:01.728652 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":34}
I20260806 21:12:01.732000 27362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd: Bootstrap starting.
I20260806 21:12:01.734408 27362 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd: Neither blocks nor log segments found. Creating new log.
I20260806 21:12:01.735105 27362 log.cc:824] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd: Log is configured to *not* fsync() on all Append() calls
I20260806 21:12:01.737066 27362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd: No bootstrap required, opened a new log
I20260806 21:12:01.739974 27362 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 43151 } }
I20260806 21:12:01.740232 27362 raft_consensus.cc:374] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:12:01.740278 27362 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66d06f2f26ad497aab505e20b1b54dfd, State: Initialized, Role: FOLLOWER
I20260806 21:12:01.740665 27362 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 43151 } }
I20260806 21:12:01.741081 27371 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 43151 } } }
I20260806 21:12:01.741200 27371 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:12:01.741771 27362 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [sys.catalog]: configured and running, proceeding with master startup.
W20260806 21:12:01.746425 27388 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 21:12:01.746521 27388 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 21:12:01.749104 27263 master_runner.cc:428] Detected that this master 66d06f2f26ad497aab505e20b1b54dfd is joining an existing cluster
I20260806 21:12:01.749161 27263 master_runner.cc:432] Initiating AddMaster RPC to add 127.16.178.61:43151
I20260806 21:12:01.759647 27366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:01.751640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36230 (local address 127.16.178.61:43151)
0806 21:12:01.751800 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:01.751804 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:01.751882 (+ 78us) server_negotiation.cc:407] Connection header received
0806 21:12:01.752260 (+ 378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:01.752262 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:01.752295 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:01.752355 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:01.752815 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:01.753526 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:01.754599 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:01.754815 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:01.754904 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:01.756147 (+ 1243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:01.756173 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:01.756176 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:01.756204 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:01.757915 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:01.758321 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:01.758324 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:01.758326 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:01.758411 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:01.758641 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:01.758645 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:01.758647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:01.758812 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:01.758900 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:01.759388 (+ 488us) server_negotiation.cc:299] Negotiation successful
0806 21:12:01.759508 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":96}
I20260806 21:12:01.759661 27399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:01.751555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.61:43151 (local address 127.0.0.1:36230)
0806 21:12:01.751800 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:01.751814 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:01.751916 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:01.752401 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:01.752403 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:01.752593 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:01.752691 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:01.752697 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:01.753657 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:01.753660 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:01.754511 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:01.754519 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:01.754919 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:01.754922 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:01.755055 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:01.755587 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:01.755598 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:01.756008 (+ 410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:12:01.758031 (+ 2023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:01.758033 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:01.758036 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:01.758232 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:01.758490 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:01.758493 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:01.758495 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:01.758564 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:01.759152 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:12:01.759154 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:12:01.759237 (+ 83us) client_negotiation.cc:769] Sending connection context
0806 21:12:01.759315 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 21:12:01.759459 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":13,"thread_start_us":77,"threads_started":1}
I20260806 21:12:01.761013 26605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:01.751292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58820 (local address 127.16.178.62:44091)
0806 21:12:01.751425 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:01.751431 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:01.751823 (+ 392us) server_negotiation.cc:407] Connection header received
0806 21:12:01.752008 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:01.752012 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:01.752059 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:01.752156 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:01.752794 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:01.753327 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:01.754473 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:01.754865 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:01.754979 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:01.757119 (+ 2140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:01.757123 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:01.757126 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:01.757154 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:01.758787 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:01.759165 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:01.759169 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:01.759171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:01.759296 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:01.759646 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:01.759650 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:01.759651 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:01.759901 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:01.759984 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:01.760716 (+ 732us) server_negotiation.cc:299] Negotiation successful
0806 21:12:01.760846 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 21:12:01.761118 27397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:01.751148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44091 (local address 127.0.0.1:58820)
0806 21:12:01.751725 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:01.751741 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:01.751906 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:01.752180 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:01.752183 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:01.752485 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:01.752655 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:01.752665 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:01.753430 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:01.753433 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:01.754330 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:01.754349 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:01.755174 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:01.755178 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:01.755642 (+ 464us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:01.756505 (+ 863us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:01.756521 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:01.756971 (+ 450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:12:01.758904 (+ 1933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:01.758907 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:01.758909 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:01.759073 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:01.759423 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:01.759426 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:01.759428 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:01.759527 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:01.760406 (+ 879us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:12:01.760409 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:12:01.760514 (+ 105us) client_negotiation.cc:769] Sending connection context
0806 21:12:01.760633 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 21:12:01.760782 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"mutex_wait_us":22,"thread_start_us":99,"threads_started":1}
I20260806 21:12:01.763563 27371 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:12:01.763738 27371 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 43151 } }
I20260806 21:12:01.764672 27371 leader_election.cc:288] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b90b03b56af14eeea8b3a169d902cf3e (127.16.178.62:44091)
I20260806 21:12:01.765097 26560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "66d06f2f26ad497aab505e20b1b54dfd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b90b03b56af14eeea8b3a169d902cf3e" is_pre_election: true
I20260806 21:12:01.765723 27286 leader_election.cc:302] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 66d06f2f26ad497aab505e20b1b54dfd; no voters: b90b03b56af14eeea8b3a169d902cf3e
I20260806 21:12:01.766010 27371 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
Aug 06 21:12:01 dist-test-slave-lgr5 krb5kdc[26480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050720, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.178.62@KRBTEST.COM for oryx/127.16.178.61@KRBTEST.COM
I20260806 21:12:01.776181 27366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:01.766748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36246 (local address 127.16.178.61:43151)
0806 21:12:01.766846 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:01.766849 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:01.766860 (+ 11us) server_negotiation.cc:407] Connection header received
0806 21:12:01.766911 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:01.766913 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:01.766953 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:01.767016 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:01.767661 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:01.768201 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:01.769055 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:01.769273 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:01.769357 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:01.771918 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:01.771924 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:01.771928 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:01.771964 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:01.773981 (+ 2017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:01.774608 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:01.774612 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:01.774614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:01.774707 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:01.775055 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:01.775059 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:01.775061 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:01.775301 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:01.775400 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:01.775845 (+ 445us) server_negotiation.cc:299] Negotiation successful
0806 21:12:01.775979 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 21:12:01.776230 27400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:01.766037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.61:43151 (local address 127.0.0.1:36246)
0806 21:12:01.766393 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:01.766439 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:01.766558 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:01.767076 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:01.767083 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:01.767422 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:01.767548 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:01.767554 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:01.768306 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:01.768309 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:01.768947 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:01.768955 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:01.769369 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:01.769373 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:01.769486 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:01.769991 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:01.770003 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:01.771725 (+ 1722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:12:01.774120 (+ 2395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:01.774137 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:01.774146 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:01.774504 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:01.774828 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:01.774832 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:01.774834 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:01.774941 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:01.775419 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:12:01.775429 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:12:01.775594 (+ 165us) client_negotiation.cc:769] Sending connection context
0806 21:12:01.775765 (+ 171us) client_negotiation.cc:240] Negotiation successful
0806 21:12:01.776012 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":106,"threads_started":1}
I20260806 21:12:01.778227 26545 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.16.178.61:43151: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd"
member_type: NON_VOTER
last_known_addr {
host: "127.16.178.61"
port: 43151
}
attrs {
promote: true
}
}
dest_uuid: "b90b03b56af14eeea8b3a169d902cf3e"
cas_config_opid_index: -1
I20260806 21:12:01.779047 26545 consensus_queue.cc:237] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: NON_VOTER last_known_addr { host: "127.16.178.61" port: 43151 } attrs { promote: true } }
I20260806 21:12:01.781183 27403 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 66d06f2f26ad497aab505e20b1b54dfd (127.16.178.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: NON_VOTER last_known_addr { host: "127.16.178.61" port: 43151 } attrs { promote: true } } }
I20260806 21:12:01.782182 27327 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [term 0 FOLLOWER]: Advancing to term 1
WARNING: no policy specified for oryx/127.16.178.60@KRBTEST.COM; defaulting to no policy
I20260806 21:12:01.783686 27327 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [term 1 FOLLOWER]: Refusing update from remote peer b90b03b56af14eeea8b3a169d902cf3e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 21:12:01.784343 27409 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: NON_VOTER last_known_addr { host: "127.16.178.61" port: 43151 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:12:01.786067 27403 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.16.178.61:43151
I20260806 21:12:01.786402 27403 sys_catalog.cc:455] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b90b03b56af14eeea8b3a169d902cf3e" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: NON_VOTER last_known_addr { host: "127.16.178.61" port: 43151 } attrs { promote: true } } }
I20260806 21:12:01.786603 27403 sys_catalog.cc:458] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e [sys.catalog]: This master's current role is: LEADER
I20260806 21:12:01.786648 27263 master.cc:537] Master@127.16.178.61:43151 shutting down...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.178.60@KRBTEST.COM" created.
W20260806 21:12:01.787730 26523 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260806 21:12:01.788931 26523 consensus_peers.cc:487] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e -> Peer 66d06f2f26ad497aab505e20b1b54dfd (127.16.178.61:43151): Couldn't send request to peer 66d06f2f26ad497aab505e20b1b54dfd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 21:12:01.793303 27263 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:12:01.793617 27263 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:12:01.793735 27263 tablet_replica.cc:330] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd: stopping tablet replica
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.178.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.60.keytab.
Entry for principal oryx/127.16.178.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.60.keytab.
I20260806 21:12:01.809667 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.60:46781
--webserver_interface=127.16.178.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.60.keytab
--principal=oryx/127.16.178.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:43991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:44091,127.16.178.61:43151,127.16.178.60:46781
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 21:12:01.810071 27263 master.cc:552] Master@127.16.178.61:43151 shutdown complete.
I20260806 21:12:01.810209 27263 master_runner.cc:305] Clearing existing system tablet
I20260806 21:12:01.810518 27263 fs_manager.cc:359] Metadata directory not provided
I20260806 21:12:01.810582 27263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:12:01.812259 27263 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 21:12:01.813220 27417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:01.813483 27263 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:12:01.813618 27263 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 21:12:01.813680 27263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-1/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-1/wal
uuid: "66d06f2f26ad497aab505e20b1b54dfd"
format_stamp: "Formatted at 2026-08-06 21:12:01 on dist-test-slave-lgr5"
server_key: "67faf9da9b1dc5013dfa799333000263"
server_key_iv: "6f4c9ccfe21854682e385287470270c5"
server_key_version: "encryptionkey@0"
I20260806 21:12:01.816205 27263 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:12:01.818944 27263 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 21:12:01.819072 27263 log.cc:1196] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260806 21:12:01.819382 27263 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd: Deleting consensus metadata
I20260806 21:12:01.819572 27263 master_runner.cc:315] Copying system tablet from 127.16.178.62:44091
I20260806 21:12:01.821252 27263 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd: tablet copy: Beginning tablet copy session from remote peer at address 127.16.178.62:44091
I20260806 21:12:01.831359 27424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:01.822440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44091 (local address 127.0.0.1:58830)
0806 21:12:01.822887 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:01.822902 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:01.823056 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:01.823342 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:01.823346 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:01.823680 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:01.823824 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:01.823832 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:01.825000 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:01.825004 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:01.825911 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:01.825922 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:01.826524 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:01.826528 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:01.826627 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:01.827219 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:01.827235 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:01.827660 (+ 425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:12:01.829651 (+ 1991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:01.829655 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:01.829657 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:01.829923 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:01.830235 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:01.830238 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:01.830241 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:01.830338 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:01.830757 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:12:01.830760 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:12:01.830876 (+ 116us) client_negotiation.cc:769] Sending connection context
0806 21:12:01.830972 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 21:12:01.831128 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":90,"threads_started":1}
I20260806 21:12:01.831362 26605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:01.822572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58830 (local address 127.16.178.62:44091)
0806 21:12:01.822730 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:01.822735 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:01.823012 (+ 277us) server_negotiation.cc:407] Connection header received
0806 21:12:01.823158 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:01.823161 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:01.823221 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:01.823311 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:01.823955 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:01.824877 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:01.826047 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:01.826406 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:01.826519 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:01.827831 (+ 1312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:01.827836 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:01.827839 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:01.827870 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:01.829518 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:01.830033 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:01.830038 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:01.830040 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:01.830141 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:01.830428 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:01.830431 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:01.830433 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:01.830666 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:01.830766 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:01.831055 (+ 289us) server_negotiation.cc:299] Negotiation successful
0806 21:12:01.831192 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 21:12:01.832436 26575 tablet_copy_service.cc:140] P b90b03b56af14eeea8b3a169d902cf3e: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 66d06f2f26ad497aab505e20b1b54dfd ({username='oryx', principal='oryx/127.16.178.61@KRBTEST.COM'} at 127.0.0.1:58830)
I20260806 21:12:01.832617 26575 tablet_copy_service.cc:161] P b90b03b56af14eeea8b3a169d902cf3e: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 66d06f2f26ad497aab505e20b1b54dfd at {username='oryx', principal='oryx/127.16.178.61@KRBTEST.COM'} at 127.0.0.1:58830: session id = 66d06f2f26ad497aab505e20b1b54dfd-00000000000000000000000000000000
I20260806 21:12:01.834659 26575 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 21:12:01.836812 27263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 21:12:01.839745 27263 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd: tablet copy: Starting download of 0 data blocks...
I20260806 21:12:01.839987 27263 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd: tablet copy: Starting download of 1 WAL segments...
I20260806 21:12:01.842535 27263 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 21:12:01.845279 26575 tablet_copy_service.cc:342] P b90b03b56af14eeea8b3a169d902cf3e: Request end of tablet copy session 66d06f2f26ad497aab505e20b1b54dfd-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.16.178.61@KRBTEST.COM'} at 127.0.0.1:58830
I20260806 21:12:01.845449 26575 tablet_copy_service.cc:433] P b90b03b56af14eeea8b3a169d902cf3e: ending tablet copy session 66d06f2f26ad497aab505e20b1b54dfd-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 66d06f2f26ad497aab505e20b1b54dfd
I20260806 21:12:01.854744 27263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:12:01.856854 27427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:01.857213 27437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:01.857793 27263 server_base.cc:1034] running on GCE node
W20260806 21:12:01.860250 27429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:01.860514 27263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:12:01.860808 27263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:12:01.861948 27263 hybrid_clock.cc:648] HybridClock initialized: now 1786050721861929 us; error 39 us; skew 500 ppm
Aug 06 21:12:01 dist-test-slave-lgr5 krb5kdc[26480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050721, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.178.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:12:01.863647 27263 init.cc:377] Logged in from keytab as oryx/127.16.178.61@KRBTEST.COM (short username oryx)
I20260806 21:12:01.864166 27263 webserver.cc:460] Webserver started at http://127.16.178.61:46875/ using document root <none> and password file <none>
I20260806 21:12:01.864375 27263 fs_manager.cc:359] Metadata directory not provided
I20260806 21:12:01.864416 27263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:12:01.865923 27263 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 21:12:01.867010 27447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:01.867260 27263 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 21:12:01.867339 27263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-1/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-1/wal
uuid: "66d06f2f26ad497aab505e20b1b54dfd"
format_stamp: "Formatted at 2026-08-06 21:12:01 on dist-test-slave-lgr5"
server_key: "67faf9da9b1dc5013dfa799333000263"
server_key_iv: "6f4c9ccfe21854682e385287470270c5"
server_key_version: "encryptionkey@0"
I20260806 21:12:01.867429 27263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:12:01.901654 27263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:12:01.903791 27263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:12:01.909814 27263 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.61:43151
I20260806 21:12:01.910238 27533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.61:43151 every 8 connection(s)
I20260806 21:12:01.910557 27263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-1/data/info.pb
I20260806 21:12:01.912285 27536 sys_catalog.cc:263] Verifying existing consensus state
I20260806 21:12:01.913097 27536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd: Bootstrap starting.
I20260806 21:12:01.923810 27536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 21:12:01.924190 27536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd: Bootstrap complete.
I20260806 21:12:01.924923 27536 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: NON_VOTER last_known_addr { host: "127.16.178.61" port: 43151 } attrs { promote: true } }
I20260806 21:12:01.925082 27536 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 66d06f2f26ad497aab505e20b1b54dfd, State: Initialized, Role: LEARNER
I20260806 21:12:01.925209 27536 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: NON_VOTER last_known_addr { host: "127.16.178.61" port: 43151 } attrs { promote: true } }
I20260806 21:12:01.925752 27540 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: NON_VOTER last_known_addr { host: "127.16.178.61" port: 43151 } attrs { promote: true } } }
I20260806 21:12:01.925866 27540 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [sys.catalog]: This master's current role is: LEARNER
I20260806 21:12:01.925921 27536 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:12:01.929888 27560 catalog_manager.cc:1259] Loaded cluster ID: 3fb66fa414d5482586f34a12c979adfa
I20260806 21:12:01.929971 27560 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd: loading cluster ID for follower catalog manager: success
I20260806 21:12:01.931681 27560 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd: acquiring CA information for follower catalog manager: success
I20260806 21:12:01.933020 27560 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 21:12:01.957410 27415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:12:01.957695 27415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:12:01.957736 27415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:12:01.961807 27415 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:12:01.961875 27415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:12:01.961894 27415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:12:01.961910 27415 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:12:01.961939 27415 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:12:01.967538 27415 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:43991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:44091,127.16.178.61:43151,127.16.178.60:46781
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.60.keytab
--principal=oryx/127.16.178.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.60:46781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.16.178.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:12:01.968787 27415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:12:01.969846 27415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 21:12:01.973038 26831 heartbeater.cc:503] Master 127.16.178.62:44091 was elected leader, sending a full tablet report...
I20260806 21:12:01.979620 27415 server_base.cc:1034] running on GCE node
W20260806 21:12:01.979588 27567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:01.979597 27570 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:01.979686 27566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:01.980453 27415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:12:01.981622 27415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:12:01.982779 27415 hybrid_clock.cc:648] HybridClock initialized: now 1786050721982757 us; error 40 us; skew 500 ppm
Aug 06 21:12:01 dist-test-slave-lgr5 krb5kdc[26480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050721, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.178.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:12:01.985634 27415 init.cc:377] Logged in from keytab as oryx/127.16.178.60@KRBTEST.COM (short username oryx)
I20260806 21:12:01.986604 27415 webserver.cc:460] Webserver started at http://127.16.178.60:46425/ using document root <none> and password file <none>
I20260806 21:12:01.987179 27415 fs_manager.cc:359] Metadata directory not provided
I20260806 21:12:01.987249 27415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:12:01.987455 27415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:12:01.989663 27415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-2/data/instance:
uuid: "2119692719ab4361bbbca6732e4d197a"
format_stamp: "Formatted at 2026-08-06 21:12:01 on dist-test-slave-lgr5"
server_key: "e3437e401fb4247655e896634287f4c4"
server_key_iv: "4acfb9a85628973eddf1febf8a97e3e9"
server_key_version: "encryptionkey@0"
I20260806 21:12:01.990279 27415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-2/wal/instance:
uuid: "2119692719ab4361bbbca6732e4d197a"
format_stamp: "Formatted at 2026-08-06 21:12:01 on dist-test-slave-lgr5"
server_key: "e3437e401fb4247655e896634287f4c4"
server_key_iv: "4acfb9a85628973eddf1febf8a97e3e9"
server_key_version: "encryptionkey@0"
I20260806 21:12:01.994630 27415 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:12:01.997476 27584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:01.998821 27415 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 21:12:01.998951 27415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-2/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-2/wal
uuid: "2119692719ab4361bbbca6732e4d197a"
format_stamp: "Formatted at 2026-08-06 21:12:01 on dist-test-slave-lgr5"
server_key: "e3437e401fb4247655e896634287f4c4"
server_key_iv: "4acfb9a85628973eddf1febf8a97e3e9"
server_key_version: "encryptionkey@0"
I20260806 21:12:01.999066 27415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:12:02.032579 27415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:12:02.036135 27415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:12:02.036353 27415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:12:02.047041 27415 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.60:46781
I20260806 21:12:02.047338 27665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.60:46781 every 8 connection(s)
I20260806 21:12:02.048372 27415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-2/data/info.pb
I20260806 21:12:02.048884 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 27415
I20260806 21:12:02.052115 27667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:12:02.056866 27667 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 21:12:02 dist-test-slave-lgr5 krb5kdc[26480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.178.60@KRBTEST.COM
Aug 06 21:12:02 dist-test-slave-lgr5 krb5kdc[26480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050721, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.178.60@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM
I20260806 21:12:02.065037 27671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.050677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54778 (local address 127.16.178.60:46781)
0806 21:12:02.051147 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:02.051168 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:02.051214 (+ 46us) server_negotiation.cc:407] Connection header received
0806 21:12:02.052104 (+ 890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:02.052132 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:02.052567 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:02.052931 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:02.053689 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.055196 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.056063 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.056762 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.056923 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.059550 (+ 2627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:02.059556 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:02.059567 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:02.059600 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:02.061448 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:02.061855 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:02.061861 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:02.061867 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:02.061988 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:02.062293 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:02.062297 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:02.062298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:02.062804 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:02.063023 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:02.063775 (+ 752us) server_negotiation.cc:299] Negotiation successful
0806 21:12:02.064066 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":139,"threads_started":1}
I20260806 21:12:02.067989 26605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.058895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58844 (local address 127.16.178.62:44091)
0806 21:12:02.059027 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:02.059032 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:02.059139 (+ 107us) server_negotiation.cc:407] Connection header received
0806 21:12:02.059274 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:02.059277 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:02.059334 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:02.059425 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:02.060248 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.060758 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.061878 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.062151 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.062257 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.064816 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:02.064819 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:02.064821 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:02.064847 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:02.066383 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:02.066854 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:02.066858 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:02.066860 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:02.066952 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:02.067187 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:02.067190 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:02.067191 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:02.067382 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:02.067475 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:02.067752 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 21:12:02.067875 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 21:12:02.068035 27672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.058614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44091 (local address 127.0.0.1:58844)
0806 21:12:02.059011 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:02.059061 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:02.059182 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:02.059477 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:02.059483 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:02.059925 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:02.060121 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:02.060131 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.060871 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.060874 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:02.061767 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:02.061776 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.062287 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.062291 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:02.062468 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.062978 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:02.062993 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:02.064634 (+ 1641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:12:02.066513 (+ 1879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:02.066521 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:02.066525 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:02.066751 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:02.067037 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:02.067039 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:02.067040 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:02.067110 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:02.067475 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:12:02.067481 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:12:02.067585 (+ 104us) client_negotiation.cc:769] Sending connection context
0806 21:12:02.067697 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 21:12:02.067888 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":85,"threads_started":1}
I20260806 21:12:02.069026 27667 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.178.61" port: 43151 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 21:12:02 dist-test-slave-lgr5 krb5kdc[26480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050721, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.178.60@KRBTEST.COM for oryx/127.16.178.61@KRBTEST.COM
I20260806 21:12:02.077133 27672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.069517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.61:43151 (local address 127.0.0.1:36252)
0806 21:12:02.069673 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:02.069685 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:02.069754 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:02.070149 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:02.070151 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:02.070276 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:02.070366 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:02.070372 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.071190 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.071192 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:02.071705 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:02.071711 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.072144 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.072147 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:02.072223 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.072681 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:02.072692 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:02.073830 (+ 1138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:12:02.075543 (+ 1713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:02.075545 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:02.075547 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:02.075736 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:02.076043 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:02.076047 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:02.076049 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:02.076257 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:02.076697 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:12:02.076700 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:12:02.076802 (+ 102us) client_negotiation.cc:769] Sending connection context
0806 21:12:02.076895 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 21:12:02.077005 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96}
I20260806 21:12:02.077222 27673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.069657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36252 (local address 127.16.178.61:43151)
0806 21:12:02.069948 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:02.069952 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:02.069972 (+ 20us) server_negotiation.cc:407] Connection header received
0806 21:12:02.070010 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:02.070011 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:02.070051 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:02.070124 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:02.070453 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.071114 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.071789 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.072030 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.072142 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.074005 (+ 1863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:02.074008 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:02.074011 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:02.074036 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:02.075422 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:02.075839 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:02.075842 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:02.075843 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:02.075932 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:02.076371 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:02.076374 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:02.076375 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:02.076583 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:02.076676 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:02.076957 (+ 281us) server_negotiation.cc:299] Negotiation successful
0806 21:12:02.077076 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":140,"threads_started":1}
I20260806 21:12:02.078037 27667 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.178.60" port: 46781 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 21:12:02 dist-test-slave-lgr5 krb5kdc[26480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050721, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.178.60@KRBTEST.COM for oryx/127.16.178.60@KRBTEST.COM
I20260806 21:12:02.085950 27672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.078531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:46781 (local address 127.0.0.1:54782)
0806 21:12:02.078660 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:02.078674 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:02.078746 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:02.078919 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:02.078921 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:02.079044 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:02.079116 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:02.079120 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.080338 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.080341 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:02.080859 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:02.080867 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.081293 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.081296 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:02.081351 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.081763 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:02.081773 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:02.082990 (+ 1217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:12:02.084593 (+ 1603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:02.084596 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:02.084597 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:02.084806 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:02.085024 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:02.085026 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:02.085027 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:02.085098 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:02.085477 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:12:02.085480 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:12:02.085565 (+ 85us) client_negotiation.cc:769] Sending connection context
0806 21:12:02.085663 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 21:12:02.085783 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 21:12:02.085974 27671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.078635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54782 (local address 127.16.178.60:46781)
0806 21:12:02.078745 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:02.078748 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:02.078760 (+ 12us) server_negotiation.cc:407] Connection header received
0806 21:12:02.078795 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:02.078797 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:02.078830 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:02.078901 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:02.079196 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.080196 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.080976 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.081209 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.081286 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.083143 (+ 1857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:02.083146 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:02.083148 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:02.083174 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:02.084487 (+ 1313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:02.084886 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:02.084888 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:02.084890 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:02.084961 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:02.085162 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:02.085163 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:02.085164 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:02.085322 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:02.085388 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:02.085702 (+ 314us) server_negotiation.cc:299] Negotiation successful
0806 21:12:02.085836 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":43}
I20260806 21:12:02.089017 27667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a: Bootstrap starting.
I20260806 21:12:02.091470 27667 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a: Neither blocks nor log segments found. Creating new log.
I20260806 21:12:02.092268 27667 log.cc:824] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a: Log is configured to *not* fsync() on all Append() calls
I20260806 21:12:02.094532 27667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a: No bootstrap required, opened a new log
I20260806 21:12:02.097831 27667 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 43151 } } peers { permanent_uuid: "2119692719ab4361bbbca6732e4d197a" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 46781 } }
I20260806 21:12:02.098052 27667 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:12:02.098110 27667 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2119692719ab4361bbbca6732e4d197a, State: Initialized, Role: FOLLOWER
I20260806 21:12:02.098544 27667 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 43151 } } peers { permanent_uuid: "2119692719ab4361bbbca6732e4d197a" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 46781 } }
I20260806 21:12:02.099050 27675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 43151 } } peers { permanent_uuid: "2119692719ab4361bbbca6732e4d197a" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 46781 } } }
I20260806 21:12:02.099172 27675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:12:02.099756 27667 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [sys.catalog]: configured and running, proceeding with master startup.
W20260806 21:12:02.105027 27690 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 21:12:02.105129 27690 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 21:12:02.108759 27415 master_runner.cc:428] Detected that this master 2119692719ab4361bbbca6732e4d197a is joining an existing cluster
I20260806 21:12:02.108846 27415 master_runner.cc:432] Initiating AddMaster RPC to add 127.16.178.60:46781
I20260806 21:12:02.119305 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 26633
I20260806 21:12:02.131873 27671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.112898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54792 (local address 127.16.178.60:46781)
0806 21:12:02.113019 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:02.113022 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:02.115754 (+ 2732us) server_negotiation.cc:407] Connection header received
0806 21:12:02.115876 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:02.115879 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:02.115926 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:02.116002 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:02.116458 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.117016 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.118001 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.118317 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.118390 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.128808 (+ 10418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:02.128811 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:02.128813 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:02.128841 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:02.130273 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:02.130702 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:02.130706 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:02.130708 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:02.130783 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:02.131029 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:02.131033 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:02.131034 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:02.131230 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:02.131319 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:02.131555 (+ 236us) server_negotiation.cc:299] Negotiation successful
0806 21:12:02.131733 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":67,"server-negotiator.queue_time_us":48}
I20260806 21:12:02.131887 27699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.112629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:46781 (local address 127.0.0.1:54792)
0806 21:12:02.115661 (+ 3032us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:02.115674 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:02.115768 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:02.116039 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:02.116041 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:02.116262 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:02.116353 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:02.116359 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.117123 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.117125 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:02.117878 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:02.117891 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.118871 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.118873 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:02.118969 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.119372 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:02.128107 (+ 8735us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:02.128609 (+ 502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:12:02.130401 (+ 1792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:02.130404 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:02.130406 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:02.130611 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:02.130865 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:02.130868 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:02.130870 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:02.130948 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:02.131318 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:12:02.131321 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:12:02.131412 (+ 91us) client_negotiation.cc:769] Sending connection context
0806 21:12:02.131505 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 21:12:02.131674 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2970,"thread_start_us":90,"threads_started":1}
I20260806 21:12:02.148118 27673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.112206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36260 (local address 127.16.178.61:43151)
0806 21:12:02.112350 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:02.112355 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:02.112664 (+ 309us) server_negotiation.cc:407] Connection header received
0806 21:12:02.112840 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:02.112842 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:02.112896 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:02.112948 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:02.113482 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.113984 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.114793 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.115118 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.115222 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.116780 (+ 1558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:02.116784 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:02.116787 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:02.116825 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:02.118525 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:02.119020 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:02.119024 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:02.119026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:02.119116 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:02.146951 (+ 27835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:02.146956 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:02.146958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:02.147246 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:02.147331 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:02.147733 (+ 402us) server_negotiation.cc:299] Negotiation successful
0806 21:12:02.147905 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 21:12:02.148681 27698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.111967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.61:43151 (local address 127.0.0.1:36260)
0806 21:12:02.112587 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:02.112600 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:02.112731 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:02.113091 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:02.113094 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:02.113264 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:02.113357 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:02.113362 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.114102 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.114105 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:02.114684 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:02.114692 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.115241 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.115245 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:02.115364 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.116133 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:02.116146 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:02.116617 (+ 471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:12:02.118625 (+ 2008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:02.118628 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:02.118631 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:02.118902 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:02.145950 (+ 27048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:02.145953 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:02.145956 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:02.146827 (+ 871us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:02.147428 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:12:02.147431 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:12:02.147543 (+ 112us) client_negotiation.cc:769] Sending connection context
0806 21:12:02.148308 (+ 765us) client_negotiation.cc:240] Negotiation successful
0806 21:12:02.148502 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":406,"threads_started":1}
I20260806 21:12:02.150772 27697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.111431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44091 (local address 127.0.0.1:58852)
0806 21:12:02.111808 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:02.111823 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:02.111942 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:02.112242 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:02.112245 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:02.112570 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:02.112734 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:02.112744 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.113880 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.113884 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:02.114800 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:02.114811 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.115367 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.115370 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:02.115468 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.146165 (+ 30697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:02.146178 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:02.146651 (+ 473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:12:02.149088 (+ 2437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:02.149091 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:02.149093 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:02.149331 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:02.149680 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:02.149684 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:02.149686 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:02.149771 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:02.150229 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:12:02.150232 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:12:02.150332 (+ 100us) client_negotiation.cc:769] Sending connection context
0806 21:12:02.150444 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 21:12:02.150596 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":185,"threads_started":1}
I20260806 21:12:02.150787 26605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.111530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58852 (local address 127.16.178.62:44091)
0806 21:12:02.111684 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:02.111689 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:02.111902 (+ 213us) server_negotiation.cc:407] Connection header received
0806 21:12:02.112048 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:02.112051 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:02.112134 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:02.112221 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:02.112905 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.113718 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.114926 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.115253 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.115343 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.146884 (+ 31541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:02.146890 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:02.146893 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:02.146941 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:02.148660 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:02.149454 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:02.149458 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:02.149460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:02.149560 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:02.149876 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:02.149879 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:02.149881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:02.150127 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:02.150230 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:02.150500 (+ 270us) server_negotiation.cc:299] Negotiation successful
0806 21:12:02.150624 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 21:12:02.153630 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.1:46257
--local_ip_for_outbound_sockets=127.16.178.1
--tserver_master_addrs=127.16.178.62:44091
--webserver_port=35755
--webserver_interface=127.16.178.1
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.1.keytab
--principal=oryx/127.16.178.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:43991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.16.178.62:44091,127.16.178.61:43151
--tserver_master_addrs=127.16.178.62:44091,127.16.178.61:43151,127.16.178.60:46781 with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 21:12:02 dist-test-slave-lgr5 krb5kdc[26480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050720, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.178.62@KRBTEST.COM for oryx/127.16.178.60@KRBTEST.COM
I20260806 21:12:02.165575 27400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.155267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:46781 (local address 127.0.0.1:54804)
0806 21:12:02.155539 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:02.155555 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:02.155645 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:02.156269 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:02.156272 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:02.156527 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:02.156655 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:02.156663 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.157714 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.157717 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:02.158515 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:02.158531 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.159280 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.159284 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:02.159393 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.159967 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:02.159983 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:02.161647 (+ 1664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:12:02.163690 (+ 2043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:02.163695 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:02.163697 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:02.163971 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:02.164343 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:02.164347 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:02.164349 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:02.164446 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:02.164973 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:12:02.164976 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:12:02.165098 (+ 122us) client_negotiation.cc:769] Sending connection context
0806 21:12:02.165211 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 21:12:02.165383 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183}
I20260806 21:12:02.165607 27671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.155791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54804 (local address 127.16.178.60:46781)
0806 21:12:02.155931 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:02.155936 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:02.155950 (+ 14us) server_negotiation.cc:407] Connection header received
0806 21:12:02.156030 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:02.156033 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:02.156137 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:02.156222 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:02.156785 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.157581 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.158659 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.159005 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.159112 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.161844 (+ 2732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:02.161849 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:02.161852 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:02.161887 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:02.163550 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:02.164075 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:02.164155 (+ 80us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:02.164157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:02.164250 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:02.164544 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:02.164548 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:02.164550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:02.164802 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:02.164904 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:02.165308 (+ 404us) server_negotiation.cc:299] Negotiation successful
0806 21:12:02.165441 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 21:12:02.167244 26545 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.16.178.60:46781: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "2119692719ab4361bbbca6732e4d197a"
member_type: NON_VOTER
last_known_addr {
host: "127.16.178.60"
port: 46781
}
attrs {
promote: true
}
}
dest_uuid: "b90b03b56af14eeea8b3a169d902cf3e"
cas_config_opid_index: 5
I20260806 21:12:02.167951 26545 consensus_queue.cc:237] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: NON_VOTER last_known_addr { host: "127.16.178.61" port: 43151 } attrs { promote: true } } peers { permanent_uuid: "2119692719ab4361bbbca6732e4d197a" member_type: NON_VOTER last_known_addr { host: "127.16.178.60" port: 46781 } attrs { promote: true } }
I20260806 21:12:02.169443 27411 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 2119692719ab4361bbbca6732e4d197a (127.16.178.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: NON_VOTER last_known_addr { host: "127.16.178.61" port: 43151 } attrs { promote: true } } peers { permanent_uuid: "2119692719ab4361bbbca6732e4d197a" member_type: NON_VOTER last_known_addr { host: "127.16.178.60" port: 46781 } attrs { promote: true } } }
I20260806 21:12:02.170972 27411 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.16.178.60:46781
I20260806 21:12:02.171168 27411 sys_catalog.cc:455] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b90b03b56af14eeea8b3a169d902cf3e" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: NON_VOTER last_known_addr { host: "127.16.178.61" port: 43151 } attrs { promote: true } } peers { permanent_uuid: "2119692719ab4361bbbca6732e4d197a" member_type: NON_VOTER last_known_addr { host: "127.16.178.60" port: 46781 } attrs { promote: true } } }
I20260806 21:12:02.171268 27411 sys_catalog.cc:458] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e [sys.catalog]: This master's current role is: LEADER
I20260806 21:12:02.171698 27415 master.cc:537] Master@127.16.178.60:46781 shutting down...
I20260806 21:12:02.172158 27621 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:12:02.173676 27621 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [term 1 FOLLOWER]: Refusing update from remote peer b90b03b56af14eeea8b3a169d902cf3e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 21:12:02.175002 27409 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2119692719ab4361bbbca6732e4d197a" member_type: NON_VOTER last_known_addr { host: "127.16.178.60" port: 46781 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 21:12:02.176994 26523 consensus_peers.cc:487] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e -> Peer 2119692719ab4361bbbca6732e4d197a (127.16.178.60:46781): Couldn't send request to peer 2119692719ab4361bbbca6732e4d197a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 21:12:02.179353 27415 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:12:02.179682 27415 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:12:02.179795 27415 tablet_replica.cc:330] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a: stopping tablet replica
I20260806 21:12:02.183060 27673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.169373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36272 (local address 127.16.178.61:43151)
0806 21:12:02.169495 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:02.169500 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:02.177888 (+ 8388us) server_negotiation.cc:407] Connection header received
0806 21:12:02.178039 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:02.178042 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:02.178113 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:02.178212 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:12:02.178605 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.179437 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.181528 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.182202 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.182285 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.182446 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:02.182826 (+ 380us) server_negotiation.cc:299] Negotiation successful
0806 21:12:02.182910 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 21:12:02.183630 27400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.169298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.61:43151 (local address 127.0.0.1:36272)
0806 21:12:02.177784 (+ 8486us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:02.177801 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:02.177917 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:02.178354 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:02.178357 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:02.178377 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:12:02.178473 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:02.178480 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.179804 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.179807 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:02.181369 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:02.181383 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.182553 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.182556 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:02.182654 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.182706 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 21:12:02.183403 (+ 697us) client_negotiation.cc:240] Negotiation successful
0806 21:12:02.183496 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8385}
I20260806 21:12:02.184360 27499 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [term 1 LEARNER]: Refusing update from remote peer b90b03b56af14eeea8b3a169d902cf3e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 21:12:02.184901 27409 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: NON_VOTER last_known_addr { host: "127.16.178.61" port: 43151 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 21:12:02.185906 27415 master.cc:552] Master@127.16.178.60:46781 shutdown complete.
I20260806 21:12:02.185971 27415 master_runner.cc:305] Clearing existing system tablet
I20260806 21:12:02.186216 27415 fs_manager.cc:359] Metadata directory not provided
I20260806 21:12:02.186266 27415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:12:02.187053 27499 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 2119692719ab4361bbbca6732e4d197a (127.16.178.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: NON_VOTER last_known_addr { host: "127.16.178.61" port: 43151 } attrs { promote: true } } peers { permanent_uuid: "2119692719ab4361bbbca6732e4d197a" member_type: NON_VOTER last_known_addr { host: "127.16.178.60" port: 46781 } attrs { promote: true } } }
I20260806 21:12:02.189687 27540 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [sys.catalog]: SysCatalogTable state changed. Reason: New leader b90b03b56af14eeea8b3a169d902cf3e. Latest consensus state: current_term: 1 leader_uuid: "b90b03b56af14eeea8b3a169d902cf3e" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: NON_VOTER last_known_addr { host: "127.16.178.61" port: 43151 } attrs { promote: true } } peers { permanent_uuid: "2119692719ab4361bbbca6732e4d197a" member_type: NON_VOTER last_known_addr { host: "127.16.178.60" port: 46781 } attrs { promote: true } } }
I20260806 21:12:02.189845 27540 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [sys.catalog]: This master's current role is: LEARNER
I20260806 21:12:02.189955 27540 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b90b03b56af14eeea8b3a169d902cf3e" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: NON_VOTER last_known_addr { host: "127.16.178.61" port: 43151 } attrs { promote: true } } peers { permanent_uuid: "2119692719ab4361bbbca6732e4d197a" member_type: NON_VOTER last_known_addr { host: "127.16.178.60" port: 46781 } attrs { promote: true } } }
I20260806 21:12:02.190068 27540 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [sys.catalog]: This master's current role is: LEARNER
I20260806 21:12:02.190578 27409 raft_consensus.cc:1050] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e: attempting to promote NON_VOTER 66d06f2f26ad497aab505e20b1b54dfd to VOTER
I20260806 21:12:02.191164 27409 consensus_queue.cc:237] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 43151 } attrs { promote: false } } peers { permanent_uuid: "2119692719ab4361bbbca6732e4d197a" member_type: NON_VOTER last_known_addr { host: "127.16.178.60" port: 46781 } attrs { promote: true } }
I20260806 21:12:02.191421 27415 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.001s
I20260806 21:12:02.192857 27708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:02.192966 27499 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [term 1 LEARNER]: Refusing update from remote peer b90b03b56af14eeea8b3a169d902cf3e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 21:12:02.193231 27415 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:12:02.193365 27415 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 21:12:02.193403 27415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-2/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-2/wal
uuid: "2119692719ab4361bbbca6732e4d197a"
format_stamp: "Formatted at 2026-08-06 21:12:01 on dist-test-slave-lgr5"
server_key: "e3437e401fb4247655e896634287f4c4"
server_key_iv: "4acfb9a85628973eddf1febf8a97e3e9"
server_key_version: "encryptionkey@0"
I20260806 21:12:02.193449 27410 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 43151 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 21:12:02.195412 27410 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 66d06f2f26ad497aab505e20b1b54dfd (127.16.178.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 43151 } attrs { promote: false } } peers { permanent_uuid: "2119692719ab4361bbbca6732e4d197a" member_type: NON_VOTER last_known_addr { host: "127.16.178.60" port: 46781 } attrs { promote: true } } }
I20260806 21:12:02.195827 27415 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:12:02.197599 27499 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 66d06f2f26ad497aab505e20b1b54dfd (127.16.178.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 43151 } attrs { promote: false } } peers { permanent_uuid: "2119692719ab4361bbbca6732e4d197a" member_type: NON_VOTER last_known_addr { host: "127.16.178.60" port: 46781 } attrs { promote: true } } }
I20260806 21:12:02.198449 27415 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 21:12:02.198549 27415 log.cc:1196] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260806 21:12:02.198807 27415 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a: Deleting consensus metadata
I20260806 21:12:02.198973 27415 master_runner.cc:315] Copying system tablet from 127.16.178.62:44091
I20260806 21:12:02.199335 27707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b90b03b56af14eeea8b3a169d902cf3e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 43151 } attrs { promote: false } } peers { permanent_uuid: "2119692719ab4361bbbca6732e4d197a" member_type: NON_VOTER last_known_addr { host: "127.16.178.60" port: 46781 } attrs { promote: true } } }
I20260806 21:12:02.199463 27707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:12:02.200927 27410 sys_catalog.cc:455] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b90b03b56af14eeea8b3a169d902cf3e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 43151 } attrs { promote: false } } peers { permanent_uuid: "2119692719ab4361bbbca6732e4d197a" member_type: NON_VOTER last_known_addr { host: "127.16.178.60" port: 46781 } attrs { promote: true } } }
I20260806 21:12:02.201100 27410 sys_catalog.cc:458] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e [sys.catalog]: This master's current role is: LEADER
I20260806 21:12:02.201473 27415 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a: tablet copy: Beginning tablet copy session from remote peer at address 127.16.178.62:44091
I20260806 21:12:02.201629 27403 sys_catalog.cc:455] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b90b03b56af14eeea8b3a169d902cf3e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 43151 } attrs { promote: false } } peers { permanent_uuid: "2119692719ab4361bbbca6732e4d197a" member_type: NON_VOTER last_known_addr { host: "127.16.178.60" port: 46781 } attrs { promote: true } } }
I20260806 21:12:02.201750 27403 sys_catalog.cc:458] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e [sys.catalog]: This master's current role is: LEADER
I20260806 21:12:02.202207 27400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.192418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:46781 (local address 127.0.0.1:54816)
0806 21:12:02.201986 (+ 9568us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:02.202105 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.60:46781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":9466}
I20260806 21:12:02.203184 27400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.202861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:46781 (local address 127.0.0.1:54820)
0806 21:12:02.203044 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:02.203104 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.60:46781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
W20260806 21:12:02.203310 26523 consensus_peers.cc:487] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e -> Peer 2119692719ab4361bbbca6732e4d197a (127.16.178.60:46781): Couldn't send request to peer 2119692719ab4361bbbca6732e4d197a. Status: Network error: Client connection negotiation failed: client connection to 127.16.178.60:46781: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 21:12:02.209666 27043 heartbeater.cc:503] Master 127.16.178.62:44091 was elected leader, sending a full tablet report...
I20260806 21:12:02.211659 26605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.202895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58858 (local address 127.16.178.62:44091)
0806 21:12:02.203041 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:02.203045 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:02.203574 (+ 529us) server_negotiation.cc:407] Connection header received
0806 21:12:02.203716 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:02.203719 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:02.203766 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:02.203833 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:02.204401 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.204960 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.205895 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.206132 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.206227 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.207516 (+ 1289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:02.207519 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:02.207521 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:02.207565 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:02.209194 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:02.210010 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:02.210015 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:02.210017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:02.210131 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:02.210503 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:02.210507 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:02.210509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:02.210872 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:02.210956 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:02.211384 (+ 428us) server_negotiation.cc:299] Negotiation successful
0806 21:12:02.211516 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 21:12:02.212281 27714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.202918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44091 (local address 127.0.0.1:58858)
0806 21:12:02.203465 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:02.203480 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:02.203617 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:02.203874 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:02.203876 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:02.204148 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:02.204276 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:02.204283 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.205085 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.205089 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:02.205760 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:02.205771 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.206253 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.206257 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:02.206370 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.206962 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:02.206976 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:02.207350 (+ 374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:12:02.209304 (+ 1954us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:02.209308 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:02.209310 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:02.209584 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:02.210255 (+ 671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:02.210259 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:02.210261 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:02.210369 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:02.211045 (+ 676us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:12:02.211048 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:12:02.211167 (+ 119us) client_negotiation.cc:769] Sending connection context
0806 21:12:02.211876 (+ 709us) client_negotiation.cc:240] Negotiation successful
0806 21:12:02.212066 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":88,"threads_started":1}
I20260806 21:12:02.212744 26575 tablet_copy_service.cc:140] P b90b03b56af14eeea8b3a169d902cf3e: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 2119692719ab4361bbbca6732e4d197a ({username='oryx', principal='oryx/127.16.178.60@KRBTEST.COM'} at 127.0.0.1:58858)
I20260806 21:12:02.212844 26575 tablet_copy_service.cc:161] P b90b03b56af14eeea8b3a169d902cf3e: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 2119692719ab4361bbbca6732e4d197a at {username='oryx', principal='oryx/127.16.178.60@KRBTEST.COM'} at 127.0.0.1:58858: session id = 2119692719ab4361bbbca6732e4d197a-00000000000000000000000000000000
I20260806 21:12:02.213814 26575 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 21:12:02.215086 27415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 21:12:02.217146 27415 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a: tablet copy: Starting download of 0 data blocks...
I20260806 21:12:02.217329 27415 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a: tablet copy: Starting download of 1 WAL segments...
I20260806 21:12:02.218703 27415 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 21:12:02.220243 26575 tablet_copy_service.cc:342] P b90b03b56af14eeea8b3a169d902cf3e: Request end of tablet copy session 2119692719ab4361bbbca6732e4d197a-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.16.178.60@KRBTEST.COM'} at 127.0.0.1:58858
I20260806 21:12:02.220326 26575 tablet_copy_service.cc:433] P b90b03b56af14eeea8b3a169d902cf3e: ending tablet copy session 2119692719ab4361bbbca6732e4d197a-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 2119692719ab4361bbbca6732e4d197a
I20260806 21:12:02.226969 27415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:12:02.228637 27715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:02.228926 27719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:02.228914 27716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:02.232630 27415 server_base.cc:1034] running on GCE node
I20260806 21:12:02.232816 27415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:12:02.233117 27415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:12:02.234261 27415 hybrid_clock.cc:648] HybridClock initialized: now 1786050722234249 us; error 31 us; skew 500 ppm
Aug 06 21:12:02 dist-test-slave-lgr5 krb5kdc[26480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050722, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.178.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:12:02.235651 27415 init.cc:377] Logged in from keytab as oryx/127.16.178.60@KRBTEST.COM (short username oryx)
I20260806 21:12:02.236142 27415 webserver.cc:460] Webserver started at http://127.16.178.60:35545/ using document root <none> and password file <none>
I20260806 21:12:02.236337 27415 fs_manager.cc:359] Metadata directory not provided
I20260806 21:12:02.236372 27415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:12:02.237468 27415 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 21:12:02.238328 27734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:02.238545 27415 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 21:12:02.238618 27415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-2/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-2/wal
uuid: "2119692719ab4361bbbca6732e4d197a"
format_stamp: "Formatted at 2026-08-06 21:12:01 on dist-test-slave-lgr5"
server_key: "e3437e401fb4247655e896634287f4c4"
server_key_iv: "4acfb9a85628973eddf1febf8a97e3e9"
server_key_version: "encryptionkey@0"
I20260806 21:12:02.238708 27415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:12:02.255810 27415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:12:02.257702 27415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:12:02.262380 27415 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.60:46781
I20260806 21:12:02.262533 27818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.60:46781 every 8 connection(s)
I20260806 21:12:02.263007 27415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/master-2/data/info.pb
I20260806 21:12:02.264724 27821 sys_catalog.cc:263] Verifying existing consensus state
I20260806 21:12:02.265648 27821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a: Bootstrap starting.
I20260806 21:12:02.275141 27821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 21:12:02.275465 27821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a: Bootstrap complete.
I20260806 21:12:02.276041 27821 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 43151 } attrs { promote: false } } peers { permanent_uuid: "2119692719ab4361bbbca6732e4d197a" member_type: NON_VOTER last_known_addr { host: "127.16.178.60" port: 46781 } attrs { promote: true } }
I20260806 21:12:02.276208 27821 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2119692719ab4361bbbca6732e4d197a, State: Initialized, Role: LEARNER
I20260806 21:12:02.276331 27821 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 43151 } attrs { promote: false } } peers { permanent_uuid: "2119692719ab4361bbbca6732e4d197a" member_type: NON_VOTER last_known_addr { host: "127.16.178.60" port: 46781 } attrs { promote: true } }
I20260806 21:12:02.276678 27826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 43151 } attrs { promote: false } } peers { permanent_uuid: "2119692719ab4361bbbca6732e4d197a" member_type: NON_VOTER last_known_addr { host: "127.16.178.60" port: 46781 } attrs { promote: true } } }
I20260806 21:12:02.276829 27821 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:12:02.276863 27826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [sys.catalog]: This master's current role is: LEARNER
I20260806 21:12:02.280699 27838 catalog_manager.cc:1259] Loaded cluster ID: 3fb66fa414d5482586f34a12c979adfa
I20260806 21:12:02.280756 27838 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a: loading cluster ID for follower catalog manager: success
I20260806 21:12:02.282428 27838 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a: acquiring CA information for follower catalog manager: success
I20260806 21:12:02.283625 27838 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 21:12:02.331884 27703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:12:02.332429 27703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:12:02.332525 27703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:12:02.338914 27703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:12:02.339026 27703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:12:02.339111 27703 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.1
I20260806 21:12:02.345211 27703 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:43991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.1.keytab
--principal=oryx/127.16.178.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.1:46257
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.178.1
--webserver_port=35755
--tserver_master_addrs=127.16.178.62:44091,127.16.178.61:43151,127.16.178.60:46781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.1
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:12:02.346513 27703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:12:02.347657 27703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:12:02.357738 27846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:02.357808 27851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:02.357769 27845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:02.357986 27703 server_base.cc:1034] running on GCE node
I20260806 21:12:02.358392 27703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:12:02.359055 27703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:12:02.360255 27703 hybrid_clock.cc:648] HybridClock initialized: now 1786050722360223 us; error 32 us; skew 500 ppm
Aug 06 21:12:02 dist-test-slave-lgr5 krb5kdc[26480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050722, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.178.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:12:02.363237 27703 init.cc:377] Logged in from keytab as oryx/127.16.178.1@KRBTEST.COM (short username oryx)
I20260806 21:12:02.364217 27703 webserver.cc:460] Webserver started at http://127.16.178.1:35755/ using document root <none> and password file <none>
I20260806 21:12:02.364781 27703 fs_manager.cc:359] Metadata directory not provided
I20260806 21:12:02.364856 27703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:12:02.369177 27703 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260806 21:12:02.371650 27862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:02.373201 27703 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:12:02.373369 27703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-0/wal
uuid: "97b7760ee07347c2990d40a6cbba0360"
format_stamp: "Formatted at 2026-08-06 21:12:00 on dist-test-slave-lgr5"
server_key: "fd42abb9789eb3d6c3be88ca1ed0e41a"
server_key_iv: "3ea1cb8bb312f5fd68814fd77beada08"
server_key_version: "encryptionkey@0"
I20260806 21:12:02.374028 27703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:12:02.388885 27703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:12:02.392355 27703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:12:02.392562 27703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:12:02.393189 27703 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:12:02.394114 27703 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:12:02.394166 27703 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:02.394222 27703 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:12:02.394248 27703 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:02.403985 27703 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.1:46257
I20260806 21:12:02.404024 28044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.1:46257 every 8 connection(s)
I20260806 21:12:02.405081 27703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
I20260806 21:12:02.414242 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 27703
I20260806 21:12:02.414482 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 26841
Aug 06 21:12:02 dist-test-slave-lgr5 krb5kdc[26480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050722, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.178.1@KRBTEST.COM for oryx/127.16.178.60@KRBTEST.COM
Aug 06 21:12:02 dist-test-slave-lgr5 krb5kdc[26480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050722, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.178.1@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM
I20260806 21:12:02.425186 26605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.409453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:43261 (local address 127.16.178.62:44091)
0806 21:12:02.409587 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:02.409591 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:02.409605 (+ 14us) server_negotiation.cc:407] Connection header received
0806 21:12:02.409652 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:02.409655 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:02.409706 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:02.409785 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:02.411155 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.411693 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.413332 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.413658 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.413760 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.420751 (+ 6991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:02.420756 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:02.420759 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:02.420792 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:02.422471 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:02.423046 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:02.423051 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:02.423052 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:02.423151 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:02.423620 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:02.423624 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:02.423625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:02.423874 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:02.423952 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:02.424864 (+ 912us) server_negotiation.cc:299] Negotiation successful
0806 21:12:02.425002 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 21:12:02.427084 28053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.409218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44091 (local address 127.16.178.1:43261)
0806 21:12:02.409460 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:02.409473 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:02.409561 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:02.409999 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:02.410006 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:02.410482 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:02.410977 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:02.410993 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.411846 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.411848 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:02.413202 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:02.413214 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.413780 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.413783 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:02.413870 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.419018 (+ 5148us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:02.419033 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:02.420495 (+ 1462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:12:02.422623 (+ 2128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:02.422629 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:02.422646 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:02.422925 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:02.423270 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:02.423273 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:02.423275 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:02.423512 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:02.424048 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:12:02.424056 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:12:02.424502 (+ 446us) client_negotiation.cc:769] Sending connection context
0806 21:12:02.425403 (+ 901us) client_negotiation.cc:240] Negotiation successful
0806 21:12:02.425830 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":160,"spinlock_wait_cycles":13440,"thread_start_us":78,"threads_started":1}
I20260806 21:12:02.427861 28052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.409019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:52279 (local address 127.16.178.60:46781)
0806 21:12:02.409276 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:02.409289 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:02.409307 (+ 18us) server_negotiation.cc:407] Connection header received
0806 21:12:02.409673 (+ 366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:02.409676 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:02.409725 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:02.409785 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:02.411121 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.412256 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.413184 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.413496 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.413601 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.423524 (+ 9923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:02.423528 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:02.423531 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:02.423564 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:02.425595 (+ 2031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:02.426097 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:02.426100 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:02.426102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:02.426195 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:02.426486 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:02.426489 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:02.426491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:02.426715 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:02.426790 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:02.427435 (+ 645us) server_negotiation.cc:299] Negotiation successful
0806 21:12:02.427682 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":89,"threads_started":1}
I20260806 21:12:02.428509 28051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.408845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:46781 (local address 127.16.178.1:52279)
0806 21:12:02.409148 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:02.409161 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:02.409519 (+ 358us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:02.409997 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:02.410005 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:02.410440 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:02.410965 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:02.410988 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.412380 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.412383 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:02.413064 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:02.413073 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.413666 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.413669 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:02.413864 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.417113 (+ 3249us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:02.417142 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:02.423325 (+ 6183us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:12:02.425736 (+ 2411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:02.425738 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:02.425741 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:02.425963 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:02.426308 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:02.426310 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:02.426312 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:02.426391 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:02.426863 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:12:02.426865 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:12:02.426959 (+ 94us) client_negotiation.cc:769] Sending connection context
0806 21:12:02.428147 (+ 1188us) client_negotiation.cc:240] Negotiation successful
0806 21:12:02.428328 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":5,"spinlock_wait_cycles":5376,"thread_start_us":96,"threads_started":1}
I20260806 21:12:02.429661 28045 heartbeater.cc:347] Connected to a master server at 127.16.178.60:46781
I20260806 21:12:02.429931 28045 heartbeater.cc:464] Registering TS with master...
I20260806 21:12:02.430486 28045 heartbeater.cc:511] Master 127.16.178.60:46781 requested a full tablet report, sending...
I20260806 21:12:02.432178 27760 ts_manager.cc:194] Registered new tserver with Master: 97b7760ee07347c2990d40a6cbba0360 (127.16.178.1:46257)
I20260806 21:12:02.434011 28046 heartbeater.cc:347] Connected to a master server at 127.16.178.62:44091
I20260806 21:12:02.434116 28046 heartbeater.cc:464] Registering TS with master...
I20260806 21:12:02.434332 28046 heartbeater.cc:511] Master 127.16.178.62:44091 requested a full tablet report, sending...
Aug 06 21:12:02 dist-test-slave-lgr5 krb5kdc[26480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050722, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.178.1@KRBTEST.COM for oryx/127.16.178.61@KRBTEST.COM
I20260806 21:12:02.435389 26545 ts_manager.cc:194] Re-registered known tserver with Master: 97b7760ee07347c2990d40a6cbba0360 (127.16.178.1:46257)
I20260806 21:12:02.436875 26545 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.178.1@KRBTEST.COM'} at 127.16.178.1:43261
I20260806 21:12:02.439525 27673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.407072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:39041 (local address 127.16.178.61:43151)
0806 21:12:02.407240 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:02.407244 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:02.408521 (+ 1277us) server_negotiation.cc:407] Connection header received
0806 21:12:02.409666 (+ 1145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:02.409670 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:02.409731 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:02.409788 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:02.411170 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.411952 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.413749 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.414094 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.418737 (+ 4643us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.435580 (+ 16843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:02.435585 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:02.435588 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:02.435624 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:02.437343 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:02.437981 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:02.437985 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:02.437987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:02.438088 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:02.438499 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:02.438503 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:02.438504 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:02.438740 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:02.438826 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:02.439186 (+ 360us) server_negotiation.cc:299] Negotiation successful
0806 21:12:02.439351 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 21:12:02.440557 28050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.407431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.61:43151 (local address 127.16.178.1:39041)
0806 21:12:02.408349 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:02.408397 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:02.409388 (+ 991us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:02.409998 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:02.410006 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:02.410492 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:02.410977 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:02.410993 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.412120 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.412123 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:02.413204 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:02.413214 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.421623 (+ 8409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.421629 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:02.421793 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.433765 (+ 11972us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:02.433782 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:02.435391 (+ 1609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:12:02.437559 (+ 2168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:02.437563 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:02.437565 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:02.437842 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:02.438269 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:02.438272 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:02.438274 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:02.438374 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:02.438897 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:12:02.438899 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:12:02.439001 (+ 102us) client_negotiation.cc:769] Sending connection context
0806 21:12:02.440164 (+ 1163us) client_negotiation.cc:240] Negotiation successful
0806 21:12:02.440359 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":716,"mutex_wait_us":120,"thread_start_us":573,"threads_started":1}
I20260806 21:12:02.441341 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.2:41469
--local_ip_for_outbound_sockets=127.16.178.2
--tserver_master_addrs=127.16.178.62:44091
--webserver_port=38187
--webserver_interface=127.16.178.2
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.2.keytab
--principal=oryx/127.16.178.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:43991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.16.178.62:44091,127.16.178.61:43151
--tserver_master_addrs=127.16.178.62:44091,127.16.178.61:43151,127.16.178.60:46781 with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 21:12:02.442873 28047 heartbeater.cc:347] Connected to a master server at 127.16.178.61:43151
I20260806 21:12:02.442984 28047 heartbeater.cc:464] Registering TS with master...
I20260806 21:12:02.443208 28047 heartbeater.cc:511] Master 127.16.178.61:43151 requested a full tablet report, sending...
I20260806 21:12:02.444499 27478 ts_manager.cc:194] Registered new tserver with Master: 97b7760ee07347c2990d40a6cbba0360 (127.16.178.1:46257)
I20260806 21:12:02.459874 27255 heartbeater.cc:503] Master 127.16.178.62:44091 was elected leader, sending a full tablet report...
W20260806 21:12:02.565392 28058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:12:02.565721 28058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:12:02.565765 28058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:12:02.570112 28058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:12:02.570200 28058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:12:02.570263 28058 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.2
I20260806 21:12:02.575857 28058 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:43991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.2.keytab
--principal=oryx/127.16.178.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.2:41469
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.178.2
--webserver_port=38187
--tserver_master_addrs=127.16.178.62:44091,127.16.178.61:43151,127.16.178.60:46781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.2
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:12:02.577111 28058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:12:02.578198 28058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 21:12:02.588364 28058 server_base.cc:1034] running on GCE node
W20260806 21:12:02.588289 28071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:02.588310 28069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:02.588332 28065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:02.588930 28058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:12:02.589615 28058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:12:02.590786 28058 hybrid_clock.cc:648] HybridClock initialized: now 1786050722590752 us; error 48 us; skew 500 ppm
Aug 06 21:12:02 dist-test-slave-lgr5 krb5kdc[26480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050722, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.178.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:12:02.593572 28058 init.cc:377] Logged in from keytab as oryx/127.16.178.2@KRBTEST.COM (short username oryx)
I20260806 21:12:02.594532 28058 webserver.cc:460] Webserver started at http://127.16.178.2:38187/ using document root <none> and password file <none>
I20260806 21:12:02.595151 28058 fs_manager.cc:359] Metadata directory not provided
I20260806 21:12:02.595240 28058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:12:02.599457 28058 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:12:02.601581 28085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:02.602837 28058 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 21:12:02.602952 28058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-1/wal
uuid: "e8a457dc20ec498da1b35c91b59c632a"
format_stamp: "Formatted at 2026-08-06 21:12:01 on dist-test-slave-lgr5"
server_key: "b6a64ee95845b2755274528745563f62"
server_key_iv: "5498fdcd56928a00942870a8ab7090d5"
server_key_version: "encryptionkey@0"
I20260806 21:12:02.603500 28058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:12:02.615128 28058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:12:02.618100 28058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:12:02.618283 28058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:12:02.618896 28058 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:12:02.619848 28058 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:12:02.619915 28058 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:02.619963 28058 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:12:02.619984 28058 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:02.628784 28058 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.2:41469
I20260806 21:12:02.628806 28262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.2:41469 every 8 connection(s)
I20260806 21:12:02.629784 28058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
I20260806 21:12:02.640122 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 28058
I20260806 21:12:02.640296 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 27051
Aug 06 21:12:02 dist-test-slave-lgr5 krb5kdc[26480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050722, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.178.2@KRBTEST.COM for oryx/127.16.178.61@KRBTEST.COM
Aug 06 21:12:02 dist-test-slave-lgr5 krb5kdc[26480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050722, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.178.2@KRBTEST.COM for oryx/127.16.178.60@KRBTEST.COM
Aug 06 21:12:02 dist-test-slave-lgr5 krb5kdc[26480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050722, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.178.2@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM
I20260806 21:12:02.654150 27673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.633648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:45703 (local address 127.16.178.61:43151)
0806 21:12:02.633761 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:02.633766 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:02.633864 (+ 98us) server_negotiation.cc:407] Connection header received
0806 21:12:02.634278 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:02.634281 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:02.634336 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:02.634387 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:02.635799 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.636374 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.637947 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.638248 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.638334 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.647906 (+ 9572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:02.647910 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:02.647912 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:02.647941 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:02.650656 (+ 2715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:02.652789 (+ 2133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:02.652793 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:02.652795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:02.652894 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:02.653206 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:02.653210 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:02.653212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:02.653453 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:02.653527 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:02.653870 (+ 343us) server_negotiation.cc:299] Negotiation successful
0806 21:12:02.654006 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 21:12:02.654376 28270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.633206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:46781 (local address 127.16.178.2:35601)
0806 21:12:02.633464 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:02.633477 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:02.634049 (+ 572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:02.634571 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:02.634578 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:02.635045 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:02.635631 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:02.635645 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.636750 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.636753 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:02.637827 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:02.637839 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.638808 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.638812 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:02.638899 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.639265 (+ 366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:02.639281 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:02.645843 (+ 6562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:12:02.649871 (+ 4028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:02.649880 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:02.649907 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:02.650963 (+ 1056us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:02.651296 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:02.651298 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:02.651300 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:02.651557 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:02.652040 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:12:02.652049 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:12:02.652475 (+ 426us) client_negotiation.cc:769] Sending connection context
0806 21:12:02.652744 (+ 269us) client_negotiation.cc:240] Negotiation successful
0806 21:12:02.653178 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":162,"spinlock_wait_cycles":2944,"thread_start_us":76,"threads_started":1}
I20260806 21:12:02.655134 28052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.633366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:35601 (local address 127.16.178.60:46781)
0806 21:12:02.633485 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:02.633489 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:02.633551 (+ 62us) server_negotiation.cc:407] Connection header received
0806 21:12:02.634244 (+ 693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:02.634247 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:02.634309 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:02.634383 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:02.635776 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.636638 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.638455 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.638687 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.638762 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.646090 (+ 7328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:02.646095 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:02.646098 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:02.646134 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:02.649273 (+ 3139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:02.651083 (+ 1810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:02.651086 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:02.651088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:02.651191 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:02.651675 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:02.651679 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:02.651681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:02.651924 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:02.652022 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:02.654805 (+ 2783us) server_negotiation.cc:299] Negotiation successful
0806 21:12:02.654962 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 21:12:02.655160 28275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.633570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.61:43151 (local address 127.16.178.2:45703)
0806 21:12:02.633784 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:02.633797 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:02.634047 (+ 250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:02.634572 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:02.634580 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:02.635048 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:02.635638 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:02.635648 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.636523 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.636526 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:02.637827 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:02.637839 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.638356 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.638360 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:02.638448 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.639166 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:02.639183 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:02.647739 (+ 8556us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:12:02.652415 (+ 4676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:02.652419 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:02.652421 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:02.652656 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:02.653010 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:02.653013 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:02.653015 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:02.653097 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:02.653601 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:12:02.653604 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:12:02.653695 (+ 91us) client_negotiation.cc:769] Sending connection context
0806 21:12:02.654805 (+ 1110us) client_negotiation.cc:240] Negotiation successful
0806 21:12:02.654960 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":87,"spinlock_wait_cycles":13056,"thread_start_us":81,"threads_started":1}
I20260806 21:12:02.655917 26605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.631567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:33545 (local address 127.16.178.62:44091)
0806 21:12:02.631712 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:02.631716 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:02.632936 (+ 1220us) server_negotiation.cc:407] Connection header received
0806 21:12:02.634278 (+ 1342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:02.634281 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:02.634336 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:02.634420 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:02.635808 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.636638 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.637741 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.638017 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.638145 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.647601 (+ 9456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:02.647608 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:02.647611 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:02.647656 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:02.650301 (+ 2645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:02.651082 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:02.651086 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:02.651088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:02.651190 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:02.651675 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:02.651679 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:02.651681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:02.652266 (+ 585us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:02.652361 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:02.655640 (+ 3279us) server_negotiation.cc:299] Negotiation successful
0806 21:12:02.655762 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 21:12:02.656635 28269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.631918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44091 (local address 127.16.178.2:33545)
0806 21:12:02.632789 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:02.632830 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:02.634047 (+ 1217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:02.634572 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:02.634580 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:02.635047 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:02.635640 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:02.635648 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.636791 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.636794 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:02.637618 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:02.637628 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.638181 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.638184 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:02.638349 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.639465 (+ 1116us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:02.639480 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:02.647362 (+ 7882us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:12:02.650461 (+ 3099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:02.650465 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:02.650467 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:02.650949 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:02.651301 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:02.651303 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:02.651305 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:02.651557 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:02.655318 (+ 3761us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:12:02.655322 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:12:02.655417 (+ 95us) client_negotiation.cc:769] Sending connection context
0806 21:12:02.656067 (+ 650us) client_negotiation.cc:240] Negotiation successful
0806 21:12:02.656238 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":684,"mutex_wait_us":216,"spinlock_wait_cycles":38144,"thread_start_us":545,"threads_started":1}
I20260806 21:12:02.671914 28265 heartbeater.cc:347] Connected to a master server at 127.16.178.62:44091
I20260806 21:12:02.671986 28266 heartbeater.cc:347] Connected to a master server at 127.16.178.61:43151
I20260806 21:12:02.672302 28266 heartbeater.cc:464] Registering TS with master...
I20260806 21:12:02.672302 28265 heartbeater.cc:464] Registering TS with master...
I20260806 21:12:02.673135 28265 heartbeater.cc:511] Master 127.16.178.62:44091 requested a full tablet report, sending...
I20260806 21:12:02.673135 28266 heartbeater.cc:511] Master 127.16.178.61:43151 requested a full tablet report, sending...
I20260806 21:12:02.674563 26545 ts_manager.cc:194] Re-registered known tserver with Master: e8a457dc20ec498da1b35c91b59c632a (127.16.178.2:41469)
I20260806 21:12:02.674707 27478 ts_manager.cc:194] Registered new tserver with Master: e8a457dc20ec498da1b35c91b59c632a (127.16.178.2:41469)
I20260806 21:12:02.675345 26545 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.178.2@KRBTEST.COM'} at 127.16.178.2:33545
I20260806 21:12:02.679033 28263 heartbeater.cc:347] Connected to a master server at 127.16.178.60:46781
I20260806 21:12:02.679162 28263 heartbeater.cc:464] Registering TS with master...
I20260806 21:12:02.679415 28263 heartbeater.cc:511] Master 127.16.178.60:46781 requested a full tablet report, sending...
I20260806 21:12:02.679486 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.3:44489
--local_ip_for_outbound_sockets=127.16.178.3
--tserver_master_addrs=127.16.178.62:44091
--webserver_port=33583
--webserver_interface=127.16.178.3
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.3.keytab
--principal=oryx/127.16.178.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:43991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.16.178.62:44091,127.16.178.61:43151
--tserver_master_addrs=127.16.178.62:44091,127.16.178.61:43151,127.16.178.60:46781 with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 21:12:02.680398 27760 ts_manager.cc:194] Registered new tserver with Master: e8a457dc20ec498da1b35c91b59c632a (127.16.178.2:41469)
I20260806 21:12:02.776130 28052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.772226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54824 (local address 127.16.178.60:46781)
0806 21:12:02.772428 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:02.772434 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:02.772579 (+ 145us) server_negotiation.cc:407] Connection header received
0806 21:12:02.772763 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:02.772766 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:02.772834 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:02.772938 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:12:02.773246 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.773795 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.774933 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.775452 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.775569 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.775759 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:02.775856 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 21:12:02.775935 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260806 21:12:02.776139 28279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.772068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:46781 (local address 127.0.0.1:54824)
0806 21:12:02.772457 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:02.772476 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:02.772640 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:02.772966 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:02.772969 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:02.772991 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:12:02.773121 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:02.773128 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.773928 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.773932 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:02.774768 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:02.774790 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.775610 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.775615 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:02.775716 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.775746 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 21:12:02.775844 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 21:12:02.775936 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":75,"threads_started":1}
I20260806 21:12:02.777120 27774 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260806 21:12:02.777667 28280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b90b03b56af14eeea8b3a169d902cf3e. Latest consensus state: current_term: 1 leader_uuid: "b90b03b56af14eeea8b3a169d902cf3e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 43151 } attrs { promote: false } } peers { permanent_uuid: "2119692719ab4361bbbca6732e4d197a" member_type: NON_VOTER last_known_addr { host: "127.16.178.60" port: 46781 } attrs { promote: true } } }
I20260806 21:12:02.777863 28280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [sys.catalog]: This master's current role is: LEARNER
W20260806 21:12:02.806452 28276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:12:02.806795 28276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:12:02.806840 28276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:12:02.811321 28276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:12:02.811437 28276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:12:02.811503 28276 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.3
I20260806 21:12:02.817667 28276 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:43991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.3.keytab
--principal=oryx/127.16.178.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.3:44489
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.178.3
--webserver_port=33583
--tserver_master_addrs=127.16.178.62:44091,127.16.178.61:43151,127.16.178.60:46781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28276
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.3
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:12:02.819046 28276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:12:02.820292 28276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 21:12:02.831075 28276 server_base.cc:1034] running on GCE node
W20260806 21:12:02.831017 28288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:02.831017 28294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:02.831053 28289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:02.831836 28276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:12:02.832576 28276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:12:02.833729 28276 hybrid_clock.cc:648] HybridClock initialized: now 1786050722833710 us; error 34 us; skew 500 ppm
Aug 06 21:12:02 dist-test-slave-lgr5 krb5kdc[26480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050722, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.178.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:12:02.836589 28276 init.cc:377] Logged in from keytab as oryx/127.16.178.3@KRBTEST.COM (short username oryx)
I20260806 21:12:02.837523 28276 webserver.cc:460] Webserver started at http://127.16.178.3:33583/ using document root <none> and password file <none>
I20260806 21:12:02.838091 28276 fs_manager.cc:359] Metadata directory not provided
I20260806 21:12:02.838177 28276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:12:02.842278 28276 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260806 21:12:02.844441 28306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:02.845743 28276 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 21:12:02.845855 28276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-2/wal
uuid: "afab74cbf9344028ad587dc2c7896c5d"
format_stamp: "Formatted at 2026-08-06 21:12:01 on dist-test-slave-lgr5"
server_key: "338fb855bd9c62c1f8b849130efd01df"
server_key_iv: "25bcf9a02583fc9dd5a02c6166d49751"
server_key_version: "encryptionkey@0"
I20260806 21:12:02.846423 28276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:12:02.872298 28276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:12:02.876068 28276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:12:02.876338 28276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:12:02.877161 28276 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:12:02.878273 28276 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:12:02.878324 28276 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:02.878371 28276 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:12:02.878398 28276 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:02.887313 28276 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.3:44489
I20260806 21:12:02.887362 28488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.3:44489 every 8 connection(s)
I20260806 21:12:02.888474 28276 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
I20260806 21:12:02.889137 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 28276
Aug 06 21:12:02 dist-test-slave-lgr5 krb5kdc[26480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050722, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.178.3@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM
Aug 06 21:12:02 dist-test-slave-lgr5 krb5kdc[26480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050722, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.178.3@KRBTEST.COM for oryx/127.16.178.60@KRBTEST.COM
I20260806 21:12:02.903405 27673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.892799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36282 (local address 127.16.178.61:43151)
0806 21:12:02.892924 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:02.892928 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:02.893568 (+ 640us) server_negotiation.cc:407] Connection header received
0806 21:12:02.893743 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:02.893746 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:02.893793 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:02.893845 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:02.894808 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.895545 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.896861 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.897514 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.897598 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.899412 (+ 1814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:02.899416 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:02.899418 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:02.899446 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:02.901282 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:02.901873 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:02.901877 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:02.901878 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:02.901974 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:02.902304 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:02.902307 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:02.902309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:02.902572 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:02.902692 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:02.903086 (+ 394us) server_negotiation.cc:299] Negotiation successful
0806 21:12:02.903212 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 21:12:02.904697 26605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.891011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58872 (local address 127.16.178.62:44091)
0806 21:12:02.891172 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:02.891177 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:02.891968 (+ 791us) server_negotiation.cc:407] Connection header received
0806 21:12:02.892538 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:02.892542 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:02.892597 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:02.892655 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:02.893795 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.894575 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.895829 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.896237 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.896316 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.898979 (+ 2663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:02.898984 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:02.898987 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:02.899033 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:02.901456 (+ 2423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:02.902169 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:02.902173 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:02.902175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:02.902270 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:02.902604 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:02.902608 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:02.902610 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:02.902848 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:02.902951 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:02.904353 (+ 1402us) server_negotiation.cc:299] Negotiation successful
0806 21:12:02.904523 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 21:12:02.908926 28504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.891017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:56681 (local address 127.16.178.62:44091)
0806 21:12:02.891542 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:02.891547 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:02.892315 (+ 768us) server_negotiation.cc:407] Connection header received
0806 21:12:02.893599 (+ 1284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:02.893602 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:02.893652 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:02.893714 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:02.895626 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.896535 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.898548 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.898922 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.899010 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.902747 (+ 3737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:02.902752 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:02.902755 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:02.902782 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:02.905484 (+ 2702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:02.906361 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:02.906365 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:02.906366 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:02.906469 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:02.907078 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:02.907081 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:02.907083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:02.907327 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:02.907699 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:02.908339 (+ 640us) server_negotiation.cc:299] Negotiation successful
0806 21:12:02.908726 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":254,"server-negotiator.queue_time_us":438,"thread_start_us":78,"threads_started":1}
I20260806 21:12:02.909749 28502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.890935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44091 (local address 127.16.178.3:56681)
0806 21:12:02.892028 (+ 1093us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:02.892223 (+ 195us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:02.893337 (+ 1114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:02.894074 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:02.894084 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:02.894640 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:02.895429 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:02.895489 (+ 60us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.896712 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.896715 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:02.898399 (+ 1684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:02.898412 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.899179 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.899183 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:02.899345 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.899860 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:02.899876 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:02.902530 (+ 2654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:12:02.905636 (+ 3106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:02.905645 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:02.905669 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:02.906237 (+ 568us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:02.906667 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:02.906670 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:02.906672 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:02.906952 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:02.907984 (+ 1032us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:12:02.907991 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:12:02.908131 (+ 140us) client_negotiation.cc:769] Sending connection context
0806 21:12:02.909142 (+ 1011us) client_negotiation.cc:240] Negotiation successful
0806 21:12:02.909297 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":840,"mutex_wait_us":232,"spinlock_wait_cycles":17536,"thread_start_us":148,"threads_started":1}
I20260806 21:12:02.909751 28510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.892121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:46781 (local address 127.16.178.3:47277)
0806 21:12:02.892793 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:02.892807 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:02.893344 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:02.894468 (+ 1124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:02.894471 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:02.894695 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:02.895429 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:02.895451 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.896683 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.896687 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:02.898394 (+ 1707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:02.898404 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.899064 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.899068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:02.899327 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.899973 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:02.899988 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:02.902530 (+ 2542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:12:02.906025 (+ 3495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:02.906029 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:02.906031 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:02.906323 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:02.906695 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:02.906698 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:02.906699 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:02.906939 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:02.907433 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:12:02.907441 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:12:02.907884 (+ 443us) client_negotiation.cc:769] Sending connection context
0806 21:12:02.908121 (+ 237us) client_negotiation.cc:240] Negotiation successful
0806 21:12:02.908553 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"mutex_wait_us":114,"spinlock_wait_cycles":17536,"thread_start_us":89,"threads_started":1}
I20260806 21:12:02.910501 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.893127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:47277 (local address 127.16.178.60:46781)
0806 21:12:02.894236 (+ 1109us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:02.894240 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:02.894252 (+ 12us) server_negotiation.cc:407] Connection header received
0806 21:12:02.894293 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:02.894296 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:02.894340 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:02.895621 (+ 1281us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:02.895737 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.896482 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.898552 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.898886 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.899678 (+ 792us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.902746 (+ 3068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:02.902750 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:02.902753 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:02.902779 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:02.904874 (+ 2095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:02.906467 (+ 1593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:02.906470 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:02.906472 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:02.906599 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:02.907072 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:02.907075 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:02.907077 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:02.907309 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:02.910034 (+ 2725us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:02.910192 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 21:12:02.910342 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1034,"thread_start_us":93,"threads_started":1}
I20260806 21:12:02.909278 26545 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58872:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:12:02.912918 26545 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:12:02.913498 28490 heartbeater.cc:347] Connected to a master server at 127.16.178.60:46781
I20260806 21:12:02.913725 28490 heartbeater.cc:464] Registering TS with master...
I20260806 21:12:02.913785 28491 heartbeater.cc:347] Connected to a master server at 127.16.178.62:44091
I20260806 21:12:02.913863 28491 heartbeater.cc:464] Registering TS with master...
I20260806 21:12:02.914381 28490 heartbeater.cc:511] Master 127.16.178.60:46781 requested a full tablet report, sending...
I20260806 21:12:02.914381 28491 heartbeater.cc:511] Master 127.16.178.62:44091 requested a full tablet report, sending...
I20260806 21:12:02.915519 26544 ts_manager.cc:194] Re-registered known tserver with Master: afab74cbf9344028ad587dc2c7896c5d (127.16.178.3:44489)
I20260806 21:12:02.915584 27760 ts_manager.cc:194] Registered new tserver with Master: afab74cbf9344028ad587dc2c7896c5d (127.16.178.3:44489)
I20260806 21:12:02.916541 26544 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.178.3@KRBTEST.COM'} at 127.16.178.3:56681
I20260806 21:12:02.916635 28052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.892141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54836 (local address 127.16.178.60:46781)
0806 21:12:02.892267 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:02.892272 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:02.900912 (+ 8640us) server_negotiation.cc:407] Connection header received
0806 21:12:02.901102 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:02.901105 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:02.901164 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:02.901217 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:02.903333 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.904131 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.910728 (+ 6597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.911060 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.911136 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.912979 (+ 1843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:02.912984 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:02.912987 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:02.913016 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:02.914636 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:02.915153 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:02.915157 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:02.915159 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:02.915259 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:02.915590 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:02.915594 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:02.915595 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:02.915841 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:02.915960 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:02.916370 (+ 410us) server_negotiation.cc:299] Negotiation successful
0806 21:12:02.916499 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
Aug 06 21:12:02 dist-test-slave-lgr5 krb5kdc[26480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050722, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.178.3@KRBTEST.COM for oryx/127.16.178.61@KRBTEST.COM
I20260806 21:12:02.922734 28281 sys_catalog.cc:455] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b90b03b56af14eeea8b3a169d902cf3e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 43151 } attrs { promote: false } } peers { permanent_uuid: "2119692719ab4361bbbca6732e4d197a" member_type: NON_VOTER last_known_addr { host: "127.16.178.60" port: 46781 } attrs { promote: true } } }
I20260806 21:12:02.922887 28281 sys_catalog.cc:458] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e [sys.catalog]: This master's current role is: LEADER
I20260806 21:12:02.926203 28506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.892508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.61:43151 (local address 127.16.178.3:38049)
0806 21:12:02.893939 (+ 1431us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:02.893952 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:02.894042 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:02.898080 (+ 4038us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:02.898084 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:02.898292 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:02.898424 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:02.898430 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.906640 (+ 8210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.906644 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:02.907465 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:02.907475 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.908228 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.908232 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:02.908325 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.908747 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:02.908760 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:02.922274 (+ 13514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:12:02.924461 (+ 2187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:02.924465 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:02.924467 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:02.924731 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:02.925067 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:02.925070 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:02.925072 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:02.925173 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:02.925660 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:12:02.925664 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:12:02.925769 (+ 105us) client_negotiation.cc:769] Sending connection context
0806 21:12:02.925865 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 21:12:02.926046 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1367,"thread_start_us":103,"threads_started":1}
I20260806 21:12:02.926820 28514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.893095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:38049 (local address 127.16.178.61:43151)
0806 21:12:02.893689 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:02.893693 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:02.896655 (+ 2962us) server_negotiation.cc:407] Connection header received
0806 21:12:02.897002 (+ 347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:02.897016 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:02.897076 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:02.905510 (+ 8434us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:02.905630 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.906415 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.907694 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.908038 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.908552 (+ 514us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.922453 (+ 13901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:02.922458 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:02.922460 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:02.922493 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:02.924307 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:02.924841 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:02.924845 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:02.924846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:02.924943 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:02.925286 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:02.925288 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:02.925290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:02.925541 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:02.926482 (+ 941us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:02.926558 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 21:12:02.926678 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":91,"threads_started":1}
I20260806 21:12:02.926982 28281 raft_consensus.cc:1050] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e: attempting to promote NON_VOTER 2119692719ab4361bbbca6732e4d197a to VOTER
I20260806 21:12:02.927299 28281 consensus_queue.cc:237] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 43151 } attrs { promote: false } } peers { permanent_uuid: "2119692719ab4361bbbca6732e4d197a" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 46781 } attrs { promote: false } }
I20260806 21:12:02.928898 28493 heartbeater.cc:347] Connected to a master server at 127.16.178.61:43151
I20260806 21:12:02.929009 28493 heartbeater.cc:464] Registering TS with master...
I20260806 21:12:02.929018 27774 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [term 1 LEARNER]: Refusing update from remote peer b90b03b56af14eeea8b3a169d902cf3e: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260806 21:12:02.929270 28493 heartbeater.cc:511] Master 127.16.178.61:43151 requested a full tablet report, sending...
I20260806 21:12:02.929502 28278 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2119692719ab4361bbbca6732e4d197a" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 46781 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260806 21:12:02.929646 27499 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [term 1 FOLLOWER]: Refusing update from remote peer b90b03b56af14eeea8b3a169d902cf3e: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260806 21:12:02.929993 27478 ts_manager.cc:194] Registered new tserver with Master: afab74cbf9344028ad587dc2c7896c5d (127.16.178.3:44489)
I20260806 21:12:02.930032 28278 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 43151 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260806 21:12:02.932120 28520 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 2119692719ab4361bbbca6732e4d197a (127.16.178.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 43151 } attrs { promote: false } } peers { permanent_uuid: "2119692719ab4361bbbca6732e4d197a" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 46781 } attrs { promote: false } } }
I20260806 21:12:02.933018 27774 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 2119692719ab4361bbbca6732e4d197a (127.16.178.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 43151 } attrs { promote: false } } peers { permanent_uuid: "2119692719ab4361bbbca6732e4d197a" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 46781 } attrs { promote: false } } }
I20260806 21:12:02.933930 28520 sys_catalog.cc:455] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b90b03b56af14eeea8b3a169d902cf3e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 43151 } attrs { promote: false } } peers { permanent_uuid: "2119692719ab4361bbbca6732e4d197a" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 46781 } attrs { promote: false } } }
I20260806 21:12:02.934036 28520 sys_catalog.cc:458] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e [sys.catalog]: This master's current role is: LEADER
I20260806 21:12:02.934137 28520 sys_catalog.cc:455] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b90b03b56af14eeea8b3a169d902cf3e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 43151 } attrs { promote: false } } peers { permanent_uuid: "2119692719ab4361bbbca6732e4d197a" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 46781 } attrs { promote: false } } }
I20260806 21:12:02.934206 28520 sys_catalog.cc:458] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e [sys.catalog]: This master's current role is: LEADER
I20260806 21:12:02.934288 28520 sys_catalog.cc:455] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b90b03b56af14eeea8b3a169d902cf3e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 43151 } attrs { promote: false } } peers { permanent_uuid: "2119692719ab4361bbbca6732e4d197a" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 46781 } attrs { promote: false } } }
I20260806 21:12:02.934357 28520 sys_catalog.cc:458] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e [sys.catalog]: This master's current role is: LEADER
I20260806 21:12:02.934993 27499 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 2119692719ab4361bbbca6732e4d197a (127.16.178.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 43151 } attrs { promote: false } } peers { permanent_uuid: "2119692719ab4361bbbca6732e4d197a" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 46781 } attrs { promote: false } } }
I20260806 21:12:02.935528 28280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b90b03b56af14eeea8b3a169d902cf3e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 43151 } attrs { promote: false } } peers { permanent_uuid: "2119692719ab4361bbbca6732e4d197a" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 46781 } attrs { promote: false } } }
I20260806 21:12:02.935631 28280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:12:02.936786 28530 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b90b03b56af14eeea8b3a169d902cf3e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 43151 } attrs { promote: false } } peers { permanent_uuid: "2119692719ab4361bbbca6732e4d197a" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 46781 } attrs { promote: false } } }
I20260806 21:12:02.936910 28530 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:12:02.944145 28279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.937983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.3:44489 (local address 127.0.0.1:53012)
0806 21:12:02.938121 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:02.938138 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:02.938223 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:02.939922 (+ 1699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:02.939925 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:02.939958 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:12:02.940138 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:02.940146 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.941834 (+ 1688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.941837 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:02.942677 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:02.942687 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.943755 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.943759 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:02.943837 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.943862 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 21:12:02.943902 (+ 40us) client_negotiation.cc:240] Negotiation successful
0806 21:12:02.943970 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":79,"spinlock_wait_cycles":9728}
I20260806 21:12:02.944700 28535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.938789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:46257 (local address 127.0.0.1:35392)
0806 21:12:02.939062 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:02.939076 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:02.939164 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:02.939922 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:02.939925 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:02.940007 (+ 82us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:12:02.940139 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:02.940146 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.941715 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.941718 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:02.942917 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:02.942927 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.943953 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.943957 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:02.944050 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.944119 (+ 69us) client_negotiation.cc:769] Sending connection context
0806 21:12:02.944172 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 21:12:02.944552 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"mutex_wait_us":64,"spinlock_wait_cycles":5120,"thread_start_us":131,"threads_started":1}
I20260806 21:12:02.944810 28531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.938157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53012 (local address 127.16.178.3:44489)
0806 21:12:02.939483 (+ 1326us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:02.939495 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:02.939513 (+ 18us) server_negotiation.cc:407] Connection header received
0806 21:12:02.939580 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:02.939586 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:02.939768 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:02.940282 (+ 514us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:12:02.940428 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.941727 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.942805 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.943600 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.944363 (+ 763us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.944517 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:02.944565 (+ 48us) server_negotiation.cc:299] Negotiation successful
0806 21:12:02.944660 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1235,"thread_start_us":123,"threads_started":1}
I20260806 21:12:02.944736 28532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.938450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.2:41469 (local address 127.0.0.1:58950)
0806 21:12:02.938716 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:02.938730 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:02.938815 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:02.939779 (+ 964us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:02.939782 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:02.939798 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:12:02.939935 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:02.939942 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.941422 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.941426 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:02.942638 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:02.942652 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.944389 (+ 1737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.944393 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:02.944481 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.944505 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 21:12:02.944551 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 21:12:02.944612 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":2,"thread_start_us":97,"threads_started":1}
I20260806 21:12:02.945480 28536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.938869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58950 (local address 127.16.178.2:41469)
0806 21:12:02.939250 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:02.939263 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:02.939287 (+ 24us) server_negotiation.cc:407] Connection header received
0806 21:12:02.939375 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:02.939381 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:02.939624 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:02.940186 (+ 562us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:12:02.940333 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.941301 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.942985 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.944252 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.944832 (+ 580us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.944981 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:02.945224 (+ 243us) server_negotiation.cc:299] Negotiation successful
0806 21:12:02.945325 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":130,"threads_started":1}
I20260806 21:12:02.945662 28537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.939144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35392 (local address 127.16.178.1:46257)
0806 21:12:02.939446 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:02.939453 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:02.939476 (+ 23us) server_negotiation.cc:407] Connection header received
0806 21:12:02.939616 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:02.939620 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:02.939768 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:02.940284 (+ 516us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:12:02.940428 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.941596 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.943044 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.943800 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.945261 (+ 1461us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.945398 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:02.945449 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 21:12:02.945540 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":111,"threads_started":1}
I20260806 21:12:02.947890 28159 tablet_service.cc:1467] Processing CreateTablet for tablet eec0bc8ce3d74179b559fe32de9dc96f (DEFAULT_TABLE table=test-table [id=ea53cdf5d6254f1786203608523685f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:12:02.947798 28381 tablet_service.cc:1467] Processing CreateTablet for tablet eec0bc8ce3d74179b559fe32de9dc96f (DEFAULT_TABLE table=test-table [id=ea53cdf5d6254f1786203608523685f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:12:02.948933 28381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eec0bc8ce3d74179b559fe32de9dc96f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:12:02.948936 28159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eec0bc8ce3d74179b559fe32de9dc96f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:12:02.950793 27938 tablet_service.cc:1467] Processing CreateTablet for tablet eec0bc8ce3d74179b559fe32de9dc96f (DEFAULT_TABLE table=test-table [id=ea53cdf5d6254f1786203608523685f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:12:02.951824 27938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eec0bc8ce3d74179b559fe32de9dc96f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:12:02.955814 28538 tablet_bootstrap.cc:492] T eec0bc8ce3d74179b559fe32de9dc96f P afab74cbf9344028ad587dc2c7896c5d: Bootstrap starting.
I20260806 21:12:02.958051 28538 tablet_bootstrap.cc:654] T eec0bc8ce3d74179b559fe32de9dc96f P afab74cbf9344028ad587dc2c7896c5d: Neither blocks nor log segments found. Creating new log.
I20260806 21:12:02.958765 28538 log.cc:824] T eec0bc8ce3d74179b559fe32de9dc96f P afab74cbf9344028ad587dc2c7896c5d: Log is configured to *not* fsync() on all Append() calls
I20260806 21:12:02.959394 28540 tablet_bootstrap.cc:492] T eec0bc8ce3d74179b559fe32de9dc96f P e8a457dc20ec498da1b35c91b59c632a: Bootstrap starting.
I20260806 21:12:02.959915 28542 tablet_bootstrap.cc:492] T eec0bc8ce3d74179b559fe32de9dc96f P 97b7760ee07347c2990d40a6cbba0360: Bootstrap starting.
I20260806 21:12:02.960942 28538 tablet_bootstrap.cc:492] T eec0bc8ce3d74179b559fe32de9dc96f P afab74cbf9344028ad587dc2c7896c5d: No bootstrap required, opened a new log
I20260806 21:12:02.961150 28538 ts_tablet_manager.cc:1397] T eec0bc8ce3d74179b559fe32de9dc96f P afab74cbf9344028ad587dc2c7896c5d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 21:12:02.961776 28540 tablet_bootstrap.cc:654] T eec0bc8ce3d74179b559fe32de9dc96f P e8a457dc20ec498da1b35c91b59c632a: Neither blocks nor log segments found. Creating new log.
I20260806 21:12:02.962155 28542 tablet_bootstrap.cc:654] T eec0bc8ce3d74179b559fe32de9dc96f P 97b7760ee07347c2990d40a6cbba0360: Neither blocks nor log segments found. Creating new log.
I20260806 21:12:02.962641 28540 log.cc:824] T eec0bc8ce3d74179b559fe32de9dc96f P e8a457dc20ec498da1b35c91b59c632a: Log is configured to *not* fsync() on all Append() calls
I20260806 21:12:02.962862 28542 log.cc:824] T eec0bc8ce3d74179b559fe32de9dc96f P 97b7760ee07347c2990d40a6cbba0360: Log is configured to *not* fsync() on all Append() calls
I20260806 21:12:02.964725 28540 tablet_bootstrap.cc:492] T eec0bc8ce3d74179b559fe32de9dc96f P e8a457dc20ec498da1b35c91b59c632a: No bootstrap required, opened a new log
I20260806 21:12:02.964725 28542 tablet_bootstrap.cc:492] T eec0bc8ce3d74179b559fe32de9dc96f P 97b7760ee07347c2990d40a6cbba0360: No bootstrap required, opened a new log
I20260806 21:12:02.964919 28540 ts_tablet_manager.cc:1397] T eec0bc8ce3d74179b559fe32de9dc96f P e8a457dc20ec498da1b35c91b59c632a: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 21:12:02.964919 28542 ts_tablet_manager.cc:1397] T eec0bc8ce3d74179b559fe32de9dc96f P 97b7760ee07347c2990d40a6cbba0360: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 21:12:02.965103 28538 raft_consensus.cc:348] T eec0bc8ce3d74179b559fe32de9dc96f P afab74cbf9344028ad587dc2c7896c5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afab74cbf9344028ad587dc2c7896c5d" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44489 } } peers { permanent_uuid: "e8a457dc20ec498da1b35c91b59c632a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 41469 } } peers { permanent_uuid: "97b7760ee07347c2990d40a6cbba0360" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 46257 } }
I20260806 21:12:02.965332 28538 raft_consensus.cc:374] T eec0bc8ce3d74179b559fe32de9dc96f P afab74cbf9344028ad587dc2c7896c5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:12:02.965384 28538 raft_consensus.cc:729] T eec0bc8ce3d74179b559fe32de9dc96f P afab74cbf9344028ad587dc2c7896c5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afab74cbf9344028ad587dc2c7896c5d, State: Initialized, Role: FOLLOWER
I20260806 21:12:02.965806 28538 consensus_queue.cc:260] T eec0bc8ce3d74179b559fe32de9dc96f P afab74cbf9344028ad587dc2c7896c5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afab74cbf9344028ad587dc2c7896c5d" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44489 } } peers { permanent_uuid: "e8a457dc20ec498da1b35c91b59c632a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 41469 } } peers { permanent_uuid: "97b7760ee07347c2990d40a6cbba0360" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 46257 } }
I20260806 21:12:02.966837 28538 ts_tablet_manager.cc:1428] T eec0bc8ce3d74179b559fe32de9dc96f P afab74cbf9344028ad587dc2c7896c5d: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 21:12:02.967041 28491 heartbeater.cc:503] Master 127.16.178.62:44091 was elected leader, sending a full tablet report...
I20260806 21:12:02.969638 28540 raft_consensus.cc:348] T eec0bc8ce3d74179b559fe32de9dc96f P e8a457dc20ec498da1b35c91b59c632a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afab74cbf9344028ad587dc2c7896c5d" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44489 } } peers { permanent_uuid: "e8a457dc20ec498da1b35c91b59c632a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 41469 } } peers { permanent_uuid: "97b7760ee07347c2990d40a6cbba0360" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 46257 } }
I20260806 21:12:02.969934 28540 raft_consensus.cc:374] T eec0bc8ce3d74179b559fe32de9dc96f P e8a457dc20ec498da1b35c91b59c632a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:12:02.970000 28540 raft_consensus.cc:729] T eec0bc8ce3d74179b559fe32de9dc96f P e8a457dc20ec498da1b35c91b59c632a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8a457dc20ec498da1b35c91b59c632a, State: Initialized, Role: FOLLOWER
I20260806 21:12:02.970563 28540 consensus_queue.cc:260] T eec0bc8ce3d74179b559fe32de9dc96f P e8a457dc20ec498da1b35c91b59c632a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afab74cbf9344028ad587dc2c7896c5d" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44489 } } peers { permanent_uuid: "e8a457dc20ec498da1b35c91b59c632a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 41469 } } peers { permanent_uuid: "97b7760ee07347c2990d40a6cbba0360" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 46257 } }
I20260806 21:12:02.970516 28542 raft_consensus.cc:348] T eec0bc8ce3d74179b559fe32de9dc96f P 97b7760ee07347c2990d40a6cbba0360 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afab74cbf9344028ad587dc2c7896c5d" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44489 } } peers { permanent_uuid: "e8a457dc20ec498da1b35c91b59c632a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 41469 } } peers { permanent_uuid: "97b7760ee07347c2990d40a6cbba0360" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 46257 } }
I20260806 21:12:02.970762 28542 raft_consensus.cc:374] T eec0bc8ce3d74179b559fe32de9dc96f P 97b7760ee07347c2990d40a6cbba0360 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:12:02.970811 28542 raft_consensus.cc:729] T eec0bc8ce3d74179b559fe32de9dc96f P 97b7760ee07347c2990d40a6cbba0360 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97b7760ee07347c2990d40a6cbba0360, State: Initialized, Role: FOLLOWER
I20260806 21:12:02.971469 28542 consensus_queue.cc:260] T eec0bc8ce3d74179b559fe32de9dc96f P 97b7760ee07347c2990d40a6cbba0360 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afab74cbf9344028ad587dc2c7896c5d" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44489 } } peers { permanent_uuid: "e8a457dc20ec498da1b35c91b59c632a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 41469 } } peers { permanent_uuid: "97b7760ee07347c2990d40a6cbba0360" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 46257 } }
I20260806 21:12:02.971803 28540 ts_tablet_manager.cc:1428] T eec0bc8ce3d74179b559fe32de9dc96f P e8a457dc20ec498da1b35c91b59c632a: Time spent starting tablet: real 0.007s user 0.003s sys 0.004s
I20260806 21:12:02.972275 28553 raft_consensus.cc:482] T eec0bc8ce3d74179b559fe32de9dc96f P e8a457dc20ec498da1b35c91b59c632a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:12:02.972433 28542 ts_tablet_manager.cc:1428] T eec0bc8ce3d74179b559fe32de9dc96f P 97b7760ee07347c2990d40a6cbba0360: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260806 21:12:02.972447 28265 heartbeater.cc:503] Master 127.16.178.62:44091 was elected leader, sending a full tablet report...
I20260806 21:12:02.972432 28553 raft_consensus.cc:504] T eec0bc8ce3d74179b559fe32de9dc96f P e8a457dc20ec498da1b35c91b59c632a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afab74cbf9344028ad587dc2c7896c5d" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44489 } } peers { permanent_uuid: "e8a457dc20ec498da1b35c91b59c632a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 41469 } } peers { permanent_uuid: "97b7760ee07347c2990d40a6cbba0360" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 46257 } }
I20260806 21:12:02.972720 28046 heartbeater.cc:503] Master 127.16.178.62:44091 was elected leader, sending a full tablet report...
I20260806 21:12:02.973587 28553 leader_election.cc:288] T eec0bc8ce3d74179b559fe32de9dc96f P e8a457dc20ec498da1b35c91b59c632a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers afab74cbf9344028ad587dc2c7896c5d (127.16.178.3:44489), 97b7760ee07347c2990d40a6cbba0360 (127.16.178.1:46257)
I20260806 21:12:02.979820 28269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.973989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.3:44489 (local address 127.16.178.2:49495)
0806 21:12:02.974111 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:02.974126 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:02.974222 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:02.975898 (+ 1676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:02.975901 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:02.975921 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:12:02.976043 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:02.976050 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.976975 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.976978 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:02.978074 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:02.978085 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.979356 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.979360 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:02.979459 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.979483 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 21:12:02.979574 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 21:12:02.979655 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260806 21:12:02.980386 28275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.974314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:46257 (local address 127.16.178.2:40185)
0806 21:12:02.974799 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:02.974816 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:02.974898 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:02.975257 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:02.975260 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:02.975281 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:12:02.975406 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:02.975414 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.976575 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.976578 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:02.977690 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:02.977700 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.979943 (+ 2243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.979947 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:02.980067 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.980126 (+ 59us) client_negotiation.cc:769] Sending connection context
0806 21:12:02.980194 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 21:12:02.980267 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"mutex_wait_us":1}
I20260806 21:12:02.980576 28531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.974777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:49495 (local address 127.16.178.3:44489)
0806 21:12:02.975396 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:02.975399 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:02.975410 (+ 11us) server_negotiation.cc:407] Connection header received
0806 21:12:02.975447 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:02.975449 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:02.975488 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:02.975551 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:12:02.976197 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.976846 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.978559 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.979106 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.979226 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.979286 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:02.980351 (+ 1065us) server_negotiation.cc:299] Negotiation successful
0806 21:12:02.980445 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537}
I20260806 21:12:02.980859 28537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:02.974901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:40185 (local address 127.16.178.1:46257)
0806 21:12:02.975008 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:02.975013 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:02.975043 (+ 30us) server_negotiation.cc:407] Connection header received
0806 21:12:02.975094 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:02.975097 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:02.975145 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:02.975219 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:12:02.975548 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.976267 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.979166 (+ 2899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:02.979717 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:02.979834 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:02.979880 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:02.980665 (+ 785us) server_negotiation.cc:299] Negotiation successful
0806 21:12:02.980745 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 21:12:02.981204 28410 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eec0bc8ce3d74179b559fe32de9dc96f" candidate_uuid: "e8a457dc20ec498da1b35c91b59c632a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afab74cbf9344028ad587dc2c7896c5d" is_pre_election: true
I20260806 21:12:02.981388 27964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eec0bc8ce3d74179b559fe32de9dc96f" candidate_uuid: "e8a457dc20ec498da1b35c91b59c632a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b7760ee07347c2990d40a6cbba0360" is_pre_election: true
I20260806 21:12:02.981594 28410 raft_consensus.cc:2454] T eec0bc8ce3d74179b559fe32de9dc96f P afab74cbf9344028ad587dc2c7896c5d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8a457dc20ec498da1b35c91b59c632a in term 0.
I20260806 21:12:02.981683 27964 raft_consensus.cc:2454] T eec0bc8ce3d74179b559fe32de9dc96f P 97b7760ee07347c2990d40a6cbba0360 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8a457dc20ec498da1b35c91b59c632a in term 0.
I20260806 21:12:02.982214 28088 leader_election.cc:302] T eec0bc8ce3d74179b559fe32de9dc96f P e8a457dc20ec498da1b35c91b59c632a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97b7760ee07347c2990d40a6cbba0360, e8a457dc20ec498da1b35c91b59c632a; no voters:
I20260806 21:12:02.982515 28553 raft_consensus.cc:2781] T eec0bc8ce3d74179b559fe32de9dc96f P e8a457dc20ec498da1b35c91b59c632a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:12:02.982615 28553 raft_consensus.cc:482] T eec0bc8ce3d74179b559fe32de9dc96f P e8a457dc20ec498da1b35c91b59c632a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:12:02.982661 28553 raft_consensus.cc:3037] T eec0bc8ce3d74179b559fe32de9dc96f P e8a457dc20ec498da1b35c91b59c632a [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:12:02.983808 28553 raft_consensus.cc:504] T eec0bc8ce3d74179b559fe32de9dc96f P e8a457dc20ec498da1b35c91b59c632a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afab74cbf9344028ad587dc2c7896c5d" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44489 } } peers { permanent_uuid: "e8a457dc20ec498da1b35c91b59c632a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 41469 } } peers { permanent_uuid: "97b7760ee07347c2990d40a6cbba0360" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 46257 } }
I20260806 21:12:02.984223 28553 leader_election.cc:288] T eec0bc8ce3d74179b559fe32de9dc96f P e8a457dc20ec498da1b35c91b59c632a [CANDIDATE]: Term 1 election: Requested vote from peers afab74cbf9344028ad587dc2c7896c5d (127.16.178.3:44489), 97b7760ee07347c2990d40a6cbba0360 (127.16.178.1:46257)
I20260806 21:12:02.984738 28410 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eec0bc8ce3d74179b559fe32de9dc96f" candidate_uuid: "e8a457dc20ec498da1b35c91b59c632a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afab74cbf9344028ad587dc2c7896c5d"
I20260806 21:12:02.984738 27964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eec0bc8ce3d74179b559fe32de9dc96f" candidate_uuid: "e8a457dc20ec498da1b35c91b59c632a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b7760ee07347c2990d40a6cbba0360"
I20260806 21:12:02.984911 28410 raft_consensus.cc:3037] T eec0bc8ce3d74179b559fe32de9dc96f P afab74cbf9344028ad587dc2c7896c5d [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:12:02.984897 27964 raft_consensus.cc:3037] T eec0bc8ce3d74179b559fe32de9dc96f P 97b7760ee07347c2990d40a6cbba0360 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:12:02.986305 28410 raft_consensus.cc:2454] T eec0bc8ce3d74179b559fe32de9dc96f P afab74cbf9344028ad587dc2c7896c5d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8a457dc20ec498da1b35c91b59c632a in term 1.
I20260806 21:12:02.986305 27964 raft_consensus.cc:2454] T eec0bc8ce3d74179b559fe32de9dc96f P 97b7760ee07347c2990d40a6cbba0360 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8a457dc20ec498da1b35c91b59c632a in term 1.
I20260806 21:12:02.986654 28088 leader_election.cc:302] T eec0bc8ce3d74179b559fe32de9dc96f P e8a457dc20ec498da1b35c91b59c632a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97b7760ee07347c2990d40a6cbba0360, e8a457dc20ec498da1b35c91b59c632a; no voters:
I20260806 21:12:02.986845 28553 raft_consensus.cc:2781] T eec0bc8ce3d74179b559fe32de9dc96f P e8a457dc20ec498da1b35c91b59c632a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:12:02.986950 28553 raft_consensus.cc:686] T eec0bc8ce3d74179b559fe32de9dc96f P e8a457dc20ec498da1b35c91b59c632a [term 1 LEADER]: Becoming Leader. State: Replica: e8a457dc20ec498da1b35c91b59c632a, State: Running, Role: LEADER
I20260806 21:12:02.987319 28553 consensus_queue.cc:237] T eec0bc8ce3d74179b559fe32de9dc96f P e8a457dc20ec498da1b35c91b59c632a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afab74cbf9344028ad587dc2c7896c5d" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44489 } } peers { permanent_uuid: "e8a457dc20ec498da1b35c91b59c632a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 41469 } } peers { permanent_uuid: "97b7760ee07347c2990d40a6cbba0360" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 46257 } }
I20260806 21:12:02.991554 26544 catalog_manager.cc:5495] T eec0bc8ce3d74179b559fe32de9dc96f P e8a457dc20ec498da1b35c91b59c632a reported cstate change: term changed from 0 to 1, leader changed from <none> to e8a457dc20ec498da1b35c91b59c632a (127.16.178.2). New cstate: current_term: 1 leader_uuid: "e8a457dc20ec498da1b35c91b59c632a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afab74cbf9344028ad587dc2c7896c5d" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8a457dc20ec498da1b35c91b59c632a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 41469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97b7760ee07347c2990d40a6cbba0360" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 46257 } health_report { overall_health: UNKNOWN } } }
I20260806 21:12:03.012012 28536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:03.008292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58954 (local address 127.16.178.2:41469)
0806 21:12:03.008434 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:03.008438 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:03.008453 (+ 15us) server_negotiation.cc:407] Connection header received
0806 21:12:03.008583 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:03.008585 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:03.008641 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:03.008733 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:12:03.008947 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:03.009478 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:03.010465 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:03.010678 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:03.010757 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:03.011095 (+ 338us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:12:03.011580 (+ 485us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:12:03.011676 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:03.011802 (+ 126us) server_negotiation.cc:299] Negotiation successful
0806 21:12:03.011879 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 21:12:03.019069 28410 raft_consensus.cc:1261] T eec0bc8ce3d74179b559fe32de9dc96f P afab74cbf9344028ad587dc2c7896c5d [term 1 FOLLOWER]: Refusing update from remote peer e8a457dc20ec498da1b35c91b59c632a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:12:03.019102 27964 raft_consensus.cc:1261] T eec0bc8ce3d74179b559fe32de9dc96f P 97b7760ee07347c2990d40a6cbba0360 [term 1 FOLLOWER]: Refusing update from remote peer e8a457dc20ec498da1b35c91b59c632a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:12:03.019843 28551 consensus_queue.cc:1035] T eec0bc8ce3d74179b559fe32de9dc96f P e8a457dc20ec498da1b35c91b59c632a [LEADER]: Connected to new peer: Peer: permanent_uuid: "97b7760ee07347c2990d40a6cbba0360" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 46257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:12:03.020098 28553 consensus_queue.cc:1035] T eec0bc8ce3d74179b559fe32de9dc96f P e8a457dc20ec498da1b35c91b59c632a [LEADER]: Connected to new peer: Peer: permanent_uuid: "afab74cbf9344028ad587dc2c7896c5d" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:12:03.027683 28561 mvcc.cc:205] Tried to move back new op lower bound from 7315663761477120000 to 7315663761359667200. Current Snapshot: MvccSnapshot[applied={T|T < 7315663761477120000}]
I20260806 21:12:03.032008 28562 mvcc.cc:205] Tried to move back new op lower bound from 7315663761477120000 to 7315663761359667200. Current Snapshot: MvccSnapshot[applied={T|T < 7315663761477120000}]
I20260806 21:12:03.038498 26544 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58872:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:12:03.038700 26544 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58872:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:12:03.044235 26544 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b90b03b56af14eeea8b3a169d902cf3e: Sending DeleteTablet for 3 replicas of tablet eec0bc8ce3d74179b559fe32de9dc96f
I20260806 21:12:03.045195 28159 tablet_service.cc:1514] Processing DeleteTablet for tablet eec0bc8ce3d74179b559fe32de9dc96f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:12:03 UTC) from {username='oryx', principal='oryx/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:58950
I20260806 21:12:03.045200 28381 tablet_service.cc:1514] Processing DeleteTablet for tablet eec0bc8ce3d74179b559fe32de9dc96f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:12:03 UTC) from {username='oryx', principal='oryx/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:53012
I20260806 21:12:03.045429 27938 tablet_service.cc:1514] Processing DeleteTablet for tablet eec0bc8ce3d74179b559fe32de9dc96f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:12:03 UTC) from {username='oryx', principal='oryx/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:35392
I20260806 21:12:03.046026 28576 tablet_replica.cc:330] T eec0bc8ce3d74179b559fe32de9dc96f P e8a457dc20ec498da1b35c91b59c632a: stopping tablet replica
I20260806 21:12:03.046309 28577 tablet_replica.cc:330] T eec0bc8ce3d74179b559fe32de9dc96f P 97b7760ee07347c2990d40a6cbba0360: stopping tablet replica
I20260806 21:12:03.046772 28577 raft_consensus.cc:2229] T eec0bc8ce3d74179b559fe32de9dc96f P 97b7760ee07347c2990d40a6cbba0360 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:12:03.046474 28575 tablet_replica.cc:330] T eec0bc8ce3d74179b559fe32de9dc96f P afab74cbf9344028ad587dc2c7896c5d: stopping tablet replica
I20260806 21:12:03.047371 28576 raft_consensus.cc:2229] T eec0bc8ce3d74179b559fe32de9dc96f P e8a457dc20ec498da1b35c91b59c632a [term 1 LEADER]: Raft consensus shutting down.
I20260806 21:12:03.047876 28576 raft_consensus.cc:2258] T eec0bc8ce3d74179b559fe32de9dc96f P e8a457dc20ec498da1b35c91b59c632a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:12:03.048159 28575 raft_consensus.cc:2229] T eec0bc8ce3d74179b559fe32de9dc96f P afab74cbf9344028ad587dc2c7896c5d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:12:03.048417 28577 raft_consensus.cc:2258] T eec0bc8ce3d74179b559fe32de9dc96f P 97b7760ee07347c2990d40a6cbba0360 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:12:03.048501 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 26494
I20260806 21:12:03.060948 28576 ts_tablet_manager.cc:1905] T eec0bc8ce3d74179b559fe32de9dc96f P e8a457dc20ec498da1b35c91b59c632a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:12:03.064407 28576 ts_tablet_manager.cc:1918] T eec0bc8ce3d74179b559fe32de9dc96f P e8a457dc20ec498da1b35c91b59c632a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:12:03.064564 28576 log.cc:1196] T eec0bc8ce3d74179b559fe32de9dc96f P e8a457dc20ec498da1b35c91b59c632a: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-1/wal/wals/eec0bc8ce3d74179b559fe32de9dc96f
I20260806 21:12:03.064918 28576 ts_tablet_manager.cc:1939] T eec0bc8ce3d74179b559fe32de9dc96f P e8a457dc20ec498da1b35c91b59c632a: Deleting consensus metadata
W20260806 21:12:03.068122 28086 connection.cc:535] server connection from 127.0.0.1:58950 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 21:12:03.068694 28575 raft_consensus.cc:2258] T eec0bc8ce3d74179b559fe32de9dc96f P afab74cbf9344028ad587dc2c7896c5d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:12:03.070329 28577 ts_tablet_manager.cc:1905] T eec0bc8ce3d74179b559fe32de9dc96f P 97b7760ee07347c2990d40a6cbba0360: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:12:03.073079 28577 ts_tablet_manager.cc:1918] T eec0bc8ce3d74179b559fe32de9dc96f P 97b7760ee07347c2990d40a6cbba0360: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:12:03.073205 28577 log.cc:1196] T eec0bc8ce3d74179b559fe32de9dc96f P 97b7760ee07347c2990d40a6cbba0360: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-0/wal/wals/eec0bc8ce3d74179b559fe32de9dc96f
I20260806 21:12:03.073591 28577 ts_tablet_manager.cc:1939] T eec0bc8ce3d74179b559fe32de9dc96f P 97b7760ee07347c2990d40a6cbba0360: Deleting consensus metadata
I20260806 21:12:03.074924 28514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:03.046750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36296 (local address 127.16.178.61:43151)
0806 21:12:03.046882 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:03.046887 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:03.059411 (+ 12524us) server_negotiation.cc:407] Connection header received
0806 21:12:03.059677 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:03.059681 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:03.059753 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:03.059809 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:03.061056 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:03.061889 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:03.063247 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:03.063650 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:03.063757 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:03.070732 (+ 6975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:03.070737 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:03.070741 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:03.070780 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:03.072655 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:03.073342 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:03.073346 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:03.073348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:03.073477 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:03.073888 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:03.073892 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:03.073894 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:03.074168 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:03.074309 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:03.074592 (+ 283us) server_negotiation.cc:299] Negotiation successful
0806 21:12:03.074740 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260806 21:12:03.075809 27864 connection.cc:441] server connection from 127.0.0.1:35392 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35392 (request call id 1) could send its response
I20260806 21:12:03.077281 28575 ts_tablet_manager.cc:1905] T eec0bc8ce3d74179b559fe32de9dc96f P afab74cbf9344028ad587dc2c7896c5d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:12:03.085604 28052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:03.047141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54848 (local address 127.16.178.60:46781)
0806 21:12:03.047259 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:03.047263 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:03.075366 (+ 28103us) server_negotiation.cc:407] Connection header received
0806 21:12:03.075563 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:03.075566 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:03.075627 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:03.075680 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:03.077088 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:03.077868 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:03.078982 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:03.079294 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:03.079398 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:03.081279 (+ 1881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:03.081283 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:03.081286 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:03.081313 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:03.083023 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:03.083500 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:03.083503 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:03.083505 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:03.083595 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:03.083889 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:03.083892 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:03.083894 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:03.084556 (+ 662us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:03.084685 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:03.085323 (+ 638us) server_negotiation.cc:299] Negotiation successful
0806 21:12:03.085444 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 21:12:03.086798 28575 ts_tablet_manager.cc:1918] T eec0bc8ce3d74179b559fe32de9dc96f P afab74cbf9344028ad587dc2c7896c5d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:12:03.086928 28575 log.cc:1196] T eec0bc8ce3d74179b559fe32de9dc96f P afab74cbf9344028ad587dc2c7896c5d: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-2/wal/wals/eec0bc8ce3d74179b559fe32de9dc96f
I20260806 21:12:03.087296 28575 ts_tablet_manager.cc:1939] T eec0bc8ce3d74179b559fe32de9dc96f P afab74cbf9344028ad587dc2c7896c5d: Deleting consensus metadata
W20260806 21:12:03.088603 28307 connection.cc:441] server connection from 127.0.0.1:53012 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53012 (request call id 1) could send its response
I20260806 21:12:03.090094 28514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:03.079022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36298 (local address 127.16.178.61:43151)
0806 21:12:03.079144 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:03.079148 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:03.079721 (+ 573us) server_negotiation.cc:407] Connection header received
0806 21:12:03.079894 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:03.079897 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:03.079962 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:03.080010 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:03.081241 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:03.082063 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:03.083272 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:03.083615 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:03.083689 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:03.086045 (+ 2356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:03.086049 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:03.086051 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:03.086079 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:03.088345 (+ 2266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:03.088886 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:03.088891 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:03.088893 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:03.088987 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:03.089253 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:03.089256 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:03.089257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:03.089435 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:03.089556 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:03.089791 (+ 235us) server_negotiation.cc:299] Negotiation successful
0806 21:12:03.089930 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 21:12:03.091068 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:03.079329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54854 (local address 127.16.178.60:46781)
0806 21:12:03.079452 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:03.079456 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:03.080125 (+ 669us) server_negotiation.cc:407] Connection header received
0806 21:12:03.080165 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:03.080168 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:03.080215 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:03.080291 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:03.081204 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:03.081958 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:03.083158 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:03.083505 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:03.083580 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:03.086416 (+ 2836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:03.086420 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:03.086423 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:03.086448 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:03.089093 (+ 2645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:03.089587 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:03.089590 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:03.089592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:03.089677 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:03.090011 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:03.090015 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:03.090017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:03.090234 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:03.090362 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:03.090783 (+ 421us) server_negotiation.cc:299] Negotiation successful
0806 21:12:03.090917 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 21:12:04.036728 28592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:04.036242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44091 (local address 127.16.178.3:59171)
0806 21:12:04.036523 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:04.036606 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":83,"threads_started":1}
I20260806 21:12:04.037075 28594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:04.036594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44091 (local address 127.16.178.2:50313)
0806 21:12:04.036865 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:04.036966 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":88,"threads_started":1}
W20260806 21:12:04.037343 28491 heartbeater.cc:644] Failed to heartbeat to 127.16.178.62:44091 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.16.178.62:44091: connect: Connection refused (error 111)
W20260806 21:12:04.037680 28265 heartbeater.cc:644] Failed to heartbeat to 127.16.178.62:44091 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.16.178.62:44091: connect: Connection refused (error 111)
I20260806 21:12:04.037955 28595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:04.037254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44091 (local address 127.16.178.1:56371)
0806 21:12:04.037728 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:04.037849 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":66,"threads_started":1}
W20260806 21:12:04.038393 28046 heartbeater.cc:644] Failed to heartbeat to 127.16.178.62:44091 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.16.178.62:44091: connect: Connection refused (error 111)
I20260806 21:12:04.544117 28599 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b90b03b56af14eeea8b3a169d902cf3e)
I20260806 21:12:04.544190 28600 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b90b03b56af14eeea8b3a169d902cf3e)
I20260806 21:12:04.544338 28600 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 43151 } attrs { promote: false } } peers { permanent_uuid: "2119692719ab4361bbbca6732e4d197a" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 46781 } attrs { promote: false } }
I20260806 21:12:04.544355 28599 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 43151 } attrs { promote: false } } peers { permanent_uuid: "2119692719ab4361bbbca6732e4d197a" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 46781 } attrs { promote: false } }
I20260806 21:12:04.544888 28600 leader_election.cc:288] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b90b03b56af14eeea8b3a169d902cf3e (127.16.178.62:44091), 2119692719ab4361bbbca6732e4d197a (127.16.178.60:46781)
I20260806 21:12:04.545403 28599 leader_election.cc:288] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b90b03b56af14eeea8b3a169d902cf3e (127.16.178.62:44091), 66d06f2f26ad497aab505e20b1b54dfd (127.16.178.61:43151)
I20260806 21:12:04.545887 28602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:04.545083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44091 (local address 127.0.0.1:59384)
0806 21:12:04.545639 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:04.545764 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":61,"threads_started":1}
W20260806 21:12:04.546097 27448 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.178.62:44091: connect: Connection refused (error 111)
I20260806 21:12:04.546525 28605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:04.545530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44091 (local address 127.0.0.1:59392)
0806 21:12:04.545833 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:04.545905 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":72,"threads_started":1}
I20260806 21:12:04.546926 28602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:04.546698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44091 (local address 127.0.0.1:59404)
0806 21:12:04.546794 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:04.546847 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 21:12:04.546900 27735 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.178.62:44091: connect: Connection refused (error 111)
W20260806 21:12:04.547241 27448 leader_election.cc:334] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b90b03b56af14eeea8b3a169d902cf3e (127.16.178.62:44091): Network error: Client connection negotiation failed: client connection to 127.16.178.62:44091: connect: Connection refused (error 111)
I20260806 21:12:04.547899 28614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:04.547536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44091 (local address 127.0.0.1:59420)
0806 21:12:04.547751 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:04.547804 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":81,"threads_started":1}
W20260806 21:12:04.548101 27735 leader_election.cc:334] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b90b03b56af14eeea8b3a169d902cf3e (127.16.178.62:44091): Network error: Client connection negotiation failed: client connection to 127.16.178.62:44091: connect: Connection refused (error 111)
I20260806 21:12:04.549803 28604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:04.545391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:46781 (local address 127.0.0.1:54860)
0806 21:12:04.545692 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:04.545708 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:04.545827 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:04.546471 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:04.546474 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:04.546502 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:12:04.546627 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:04.546633 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:04.547659 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:04.547662 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:04.548676 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:04.548687 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:04.549404 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:04.549407 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:04.549519 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:04.549544 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 21:12:04.549605 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 21:12:04.549662 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"spinlock_wait_cycles":384,"thread_start_us":78,"threads_started":1}
I20260806 21:12:04.549898 28609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:04.546039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54860 (local address 127.16.178.60:46781)
0806 21:12:04.546211 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:04.546216 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:04.546229 (+ 13us) server_negotiation.cc:407] Connection header received
0806 21:12:04.546280 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:04.546283 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:04.546341 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:04.546722 (+ 381us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:12:04.546822 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:04.547547 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:04.548796 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:04.549293 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:04.549397 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:04.549460 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:04.549637 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 21:12:04.549713 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"spinlock_wait_cycles":2176,"thread_start_us":52,"threads_started":1}
I20260806 21:12:04.550263 28606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:04.545861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.61:43151 (local address 127.0.0.1:36314)
0806 21:12:04.546229 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:04.546243 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:04.546327 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:04.546824 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:04.546826 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:04.546841 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:12:04.546906 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:04.546911 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:04.547931 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:04.547934 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:04.548945 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:04.548957 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:04.549752 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:04.549756 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:04.549843 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:04.549867 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 21:12:04.550057 (+ 190us) client_negotiation.cc:240] Negotiation successful
0806 21:12:04.550130 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"spinlock_wait_cycles":4608,"thread_start_us":62,"threads_started":1}
I20260806 21:12:04.550263 28610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:04.546068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36314 (local address 127.16.178.61:43151)
0806 21:12:04.546296 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:04.546300 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:04.546317 (+ 17us) server_negotiation.cc:407] Connection header received
0806 21:12:04.546474 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:04.546477 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:04.546528 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:04.546735 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:12:04.547130 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:04.547824 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:04.549064 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:04.549633 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:04.549735 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:04.549790 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:04.550081 (+ 291us) server_negotiation.cc:299] Negotiation successful
0806 21:12:04.550154 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"spinlock_wait_cycles":6784,"thread_start_us":66,"threads_started":1}
I20260806 21:12:04.550334 27774 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "66d06f2f26ad497aab505e20b1b54dfd" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2119692719ab4361bbbca6732e4d197a" is_pre_election: true
I20260806 21:12:04.550585 27774 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66d06f2f26ad497aab505e20b1b54dfd in term 1.
I20260806 21:12:04.550752 27499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2119692719ab4361bbbca6732e4d197a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "66d06f2f26ad497aab505e20b1b54dfd" is_pre_election: true
I20260806 21:12:04.550969 27499 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2119692719ab4361bbbca6732e4d197a in term 1.
I20260806 21:12:04.551071 27448 leader_election.cc:302] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2119692719ab4361bbbca6732e4d197a, 66d06f2f26ad497aab505e20b1b54dfd; no voters: b90b03b56af14eeea8b3a169d902cf3e
I20260806 21:12:04.551246 28600 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 21:12:04.551318 28600 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [term 1 FOLLOWER]: Starting leader election (detected failure of leader b90b03b56af14eeea8b3a169d902cf3e)
I20260806 21:12:04.551362 28600 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [term 1 FOLLOWER]: Advancing to term 2
I20260806 21:12:04.552320 27735 leader_election.cc:302] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2119692719ab4361bbbca6732e4d197a, 66d06f2f26ad497aab505e20b1b54dfd; no voters: b90b03b56af14eeea8b3a169d902cf3e
I20260806 21:12:04.552551 28600 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 43151 } attrs { promote: false } } peers { permanent_uuid: "2119692719ab4361bbbca6732e4d197a" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 46781 } attrs { promote: false } }
I20260806 21:12:04.552600 28599 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 21:12:04.552657 28599 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [term 1 FOLLOWER]: Starting leader election (detected failure of leader b90b03b56af14eeea8b3a169d902cf3e)
I20260806 21:12:04.552706 28599 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [term 1 FOLLOWER]: Advancing to term 2
I20260806 21:12:04.552888 28600 leader_election.cc:288] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [CANDIDATE]: Term 2 election: Requested vote from peers b90b03b56af14eeea8b3a169d902cf3e (127.16.178.62:44091), 2119692719ab4361bbbca6732e4d197a (127.16.178.60:46781)
I20260806 21:12:04.553354 28604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:04.553053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44091 (local address 127.0.0.1:59424)
0806 21:12:04.553218 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:04.553276 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260806 21:12:04.553462 27774 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "66d06f2f26ad497aab505e20b1b54dfd" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2119692719ab4361bbbca6732e4d197a"
I20260806 21:12:04.553874 28599 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 43151 } attrs { promote: false } } peers { permanent_uuid: "2119692719ab4361bbbca6732e4d197a" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 46781 } attrs { promote: false } }
I20260806 21:12:04.553944 28604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:04.553755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44091 (local address 127.0.0.1:59438)
0806 21:12:04.553847 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:04.553895 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 21:12:04.554041 27448 leader_election.cc:334] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer b90b03b56af14eeea8b3a169d902cf3e (127.16.178.62:44091): Network error: Client connection negotiation failed: client connection to 127.16.178.62:44091: connect: Connection refused (error 111)
I20260806 21:12:04.554092 27774 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 66d06f2f26ad497aab505e20b1b54dfd in current term 2: Already voted for candidate 2119692719ab4361bbbca6732e4d197a in this term.
I20260806 21:12:04.554237 28599 leader_election.cc:288] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [CANDIDATE]: Term 2 election: Requested vote from peers b90b03b56af14eeea8b3a169d902cf3e (127.16.178.62:44091), 66d06f2f26ad497aab505e20b1b54dfd (127.16.178.61:43151)
I20260806 21:12:04.554539 28606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:04.554315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44091 (local address 127.0.0.1:59452)
0806 21:12:04.554426 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:04.554477 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 21:12:04.554543 27448 leader_election.cc:302] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 66d06f2f26ad497aab505e20b1b54dfd; no voters: 2119692719ab4361bbbca6732e4d197a, b90b03b56af14eeea8b3a169d902cf3e
I20260806 21:12:04.554694 28600 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260806 21:12:04.554755 27499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2119692719ab4361bbbca6732e4d197a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "66d06f2f26ad497aab505e20b1b54dfd"
I20260806 21:12:04.554913 27499 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 2119692719ab4361bbbca6732e4d197a in current term 2: Already voted for candidate 66d06f2f26ad497aab505e20b1b54dfd in this term.
I20260806 21:12:04.555562 28606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:04.555362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44091 (local address 127.0.0.1:59462)
0806 21:12:04.555466 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:04.555513 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260806 21:12:04.555697 27735 leader_election.cc:334] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer b90b03b56af14eeea8b3a169d902cf3e (127.16.178.62:44091): Network error: Client connection negotiation failed: client connection to 127.16.178.62:44091: connect: Connection refused (error 111)
I20260806 21:12:04.555763 27735 leader_election.cc:302] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2119692719ab4361bbbca6732e4d197a; no voters: 66d06f2f26ad497aab505e20b1b54dfd, b90b03b56af14eeea8b3a169d902cf3e
I20260806 21:12:04.555888 28599 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260806 21:12:05.038353 28617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:05.037900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44091 (local address 127.16.178.3:51231)
0806 21:12:05.038175 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:05.038237 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":100,"threads_started":1}
I20260806 21:12:05.038982 28618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:05.038434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44091 (local address 127.16.178.2:48397)
0806 21:12:05.038769 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:05.038856 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":71,"threads_started":1}
I20260806 21:12:05.039600 28619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:05.038871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44091 (local address 127.16.178.1:57101)
0806 21:12:05.039224 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:05.039506 (+ 282us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":74,"threads_started":1}
I20260806 21:12:06.039824 28620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:06.039382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44091 (local address 127.16.178.3:41147)
0806 21:12:06.039631 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:06.039703 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":74,"threads_started":1}
I20260806 21:12:06.040414 28623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:06.039860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44091 (local address 127.16.178.2:46605)
0806 21:12:06.040259 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:06.040322 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":86,"threads_started":1}
I20260806 21:12:06.041059 28624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:06.040338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44091 (local address 127.16.178.1:47615)
0806 21:12:06.040907 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:06.040967 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":67,"threads_started":1}
I20260806 21:12:06.132901 28625 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:12:06.133059 28625 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 43151 } attrs { promote: false } } peers { permanent_uuid: "2119692719ab4361bbbca6732e4d197a" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 46781 } attrs { promote: false } }
I20260806 21:12:06.133494 28625 leader_election.cc:288] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b90b03b56af14eeea8b3a169d902cf3e (127.16.178.62:44091), 66d06f2f26ad497aab505e20b1b54dfd (127.16.178.61:43151)
I20260806 21:12:06.134157 28627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:06.133722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44091 (local address 127.0.0.1:59674)
0806 21:12:06.133970 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:06.134051 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":65,"threads_started":1}
I20260806 21:12:06.134356 27499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2119692719ab4361bbbca6732e4d197a" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "66d06f2f26ad497aab505e20b1b54dfd" is_pre_election: true
I20260806 21:12:06.134531 27499 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2119692719ab4361bbbca6732e4d197a in term 2.
I20260806 21:12:06.134907 27735 leader_election.cc:302] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2119692719ab4361bbbca6732e4d197a, 66d06f2f26ad497aab505e20b1b54dfd; no voters:
I20260806 21:12:06.135004 28627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:06.134742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44091 (local address 127.0.0.1:59676)
0806 21:12:06.134848 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:06.134902 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 21:12:06.135097 27735 leader_election.cc:334] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b90b03b56af14eeea8b3a169d902cf3e (127.16.178.62:44091): Network error: Client connection negotiation failed: client connection to 127.16.178.62:44091: connect: Connection refused (error 111)
I20260806 21:12:06.135135 28625 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260806 21:12:06.135205 28625 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:12:06.135248 28625 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [term 2 FOLLOWER]: Advancing to term 3
I20260806 21:12:06.136474 28625 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 43151 } attrs { promote: false } } peers { permanent_uuid: "2119692719ab4361bbbca6732e4d197a" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 46781 } attrs { promote: false } }
I20260806 21:12:06.136843 28625 leader_election.cc:288] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [CANDIDATE]: Term 3 election: Requested vote from peers b90b03b56af14eeea8b3a169d902cf3e (127.16.178.62:44091), 66d06f2f26ad497aab505e20b1b54dfd (127.16.178.61:43151)
I20260806 21:12:06.137264 28627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:06.137016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44091 (local address 127.0.0.1:59688)
0806 21:12:06.137133 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:06.137189 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 21:12:06.137524 27499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2119692719ab4361bbbca6732e4d197a" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "66d06f2f26ad497aab505e20b1b54dfd"
I20260806 21:12:06.137725 27499 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [term 2 FOLLOWER]: Advancing to term 3
I20260806 21:12:06.137912 28627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:06.137697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44091 (local address 127.0.0.1:59696)
0806 21:12:06.137798 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:06.137851 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260806 21:12:06.138013 27735 leader_election.cc:334] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer b90b03b56af14eeea8b3a169d902cf3e (127.16.178.62:44091): Network error: Client connection negotiation failed: client connection to 127.16.178.62:44091: connect: Connection refused (error 111)
I20260806 21:12:06.138852 27499 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2119692719ab4361bbbca6732e4d197a in term 3.
I20260806 21:12:06.139216 27735 leader_election.cc:302] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2119692719ab4361bbbca6732e4d197a, 66d06f2f26ad497aab505e20b1b54dfd; no voters: b90b03b56af14eeea8b3a169d902cf3e
I20260806 21:12:06.139418 28625 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [term 3 FOLLOWER]: Leader election won for term 3
I20260806 21:12:06.139657 28625 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [term 3 LEADER]: Becoming Leader. State: Replica: 2119692719ab4361bbbca6732e4d197a, State: Running, Role: LEADER
I20260806 21:12:06.139937 28625 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 43151 } attrs { promote: false } } peers { permanent_uuid: "2119692719ab4361bbbca6732e4d197a" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 46781 } attrs { promote: false } }
I20260806 21:12:06.141537 28629 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2119692719ab4361bbbca6732e4d197a. Latest consensus state: current_term: 3 leader_uuid: "2119692719ab4361bbbca6732e4d197a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 43151 } attrs { promote: false } } peers { permanent_uuid: "2119692719ab4361bbbca6732e4d197a" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 46781 } attrs { promote: false } } }
I20260806 21:12:06.141698 28629 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [sys.catalog]: This master's current role is: LEADER
I20260806 21:12:06.142058 28632 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:12:06.144763 28632 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:12:06.145171 28632 catalog_manager.cc:1259] Loaded cluster ID: 3fb66fa414d5482586f34a12c979adfa
I20260806 21:12:06.145219 28632 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:12:06.145807 28632 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:12:06.146205 28632 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a: Loaded TSK: 0
I20260806 21:12:06.146895 28632 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:12:06.180579 27757 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54854:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:12:06.183707 27757 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:12:06.187036 28627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:06.186780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44091 (local address 127.0.0.1:59710)
0806 21:12:06.186897 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:06.186956 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 21:12:06.187367 27499 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [term 3 FOLLOWER]: Refusing update from remote peer 2119692719ab4361bbbca6732e4d197a: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260806 21:12:06.187738 28627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:06.187534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44091 (local address 127.0.0.1:59714)
0806 21:12:06.187629 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:06.187678 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260806 21:12:06.187875 27735 consensus_peers.cc:487] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a -> Peer b90b03b56af14eeea8b3a169d902cf3e (127.16.178.62:44091): Couldn't send request to peer b90b03b56af14eeea8b3a169d902cf3e. Status: Network error: Client connection negotiation failed: client connection to 127.16.178.62:44091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 21:12:06.188169 28625 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [LEADER]: Connected to new peer: Peer: permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 43151 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260806 21:12:06.189821 28634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2119692719ab4361bbbca6732e4d197a. Latest consensus state: current_term: 3 leader_uuid: "2119692719ab4361bbbca6732e4d197a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 43151 } attrs { promote: false } } peers { permanent_uuid: "2119692719ab4361bbbca6732e4d197a" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 46781 } attrs { promote: false } } }
I20260806 21:12:06.189989 28634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:12:06.191222 28625 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "2119692719ab4361bbbca6732e4d197a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 43151 } attrs { promote: false } } peers { permanent_uuid: "2119692719ab4361bbbca6732e4d197a" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 46781 } attrs { promote: false } } }
I20260806 21:12:06.191367 28625 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a [sys.catalog]: This master's current role is: LEADER
I20260806 21:12:06.191687 28634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "2119692719ab4361bbbca6732e4d197a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b90b03b56af14eeea8b3a169d902cf3e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44091 } } peers { permanent_uuid: "66d06f2f26ad497aab505e20b1b54dfd" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 43151 } attrs { promote: false } } peers { permanent_uuid: "2119692719ab4361bbbca6732e4d197a" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 46781 } attrs { promote: false } } }
I20260806 21:12:06.191978 28634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66d06f2f26ad497aab505e20b1b54dfd [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:12:06.191784 28635 mvcc.cc:205] Tried to move back new op lower bound from 7315663774457110528 to 7315663774272086016. Current Snapshot: MvccSnapshot[applied={T|T < 7315663774457110528}]
I20260806 21:12:06.202302 28627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:06.197419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.3:44489 (local address 127.0.0.1:53014)
0806 21:12:06.197571 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:06.197590 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:06.197693 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:06.198628 (+ 935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:06.198632 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:06.198650 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:12:06.198800 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:06.198807 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:06.199981 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:06.199985 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:06.200987 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:06.200997 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:06.201901 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:06.201904 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:06.202005 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:06.202032 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 21:12:06.202079 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 21:12:06.202152 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65,"mutex_wait_us":20}
I20260806 21:12:06.202456 28643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:06.198073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58956 (local address 127.16.178.2:41469)
0806 21:12:06.198563 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:06.198569 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:06.198584 (+ 15us) server_negotiation.cc:407] Connection header received
0806 21:12:06.198679 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:06.198683 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:06.198732 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:06.198811 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:12:06.199063 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:06.199625 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:06.201250 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:06.201709 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:06.201828 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:06.201907 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:06.202220 (+ 313us) server_negotiation.cc:299] Negotiation successful
0806 21:12:06.202311 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":91,"threads_started":1}
I20260806 21:12:06.202476 28644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:06.198279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35400 (local address 127.16.178.1:46257)
0806 21:12:06.198509 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:06.198513 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:06.198527 (+ 14us) server_negotiation.cc:407] Connection header received
0806 21:12:06.198579 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:06.198582 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:06.198627 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:06.198985 (+ 358us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:12:06.199091 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:06.199835 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:06.201322 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:06.201915 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:06.202008 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:06.202070 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:06.202223 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 21:12:06.202344 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":74,"threads_started":1}
I20260806 21:12:06.202708 28642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:06.197967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.2:41469 (local address 127.0.0.1:58956)
0806 21:12:06.198492 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:06.198505 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:06.198588 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:06.198829 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:06.198832 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:06.198849 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:12:06.198951 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:06.198958 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:06.199740 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:06.199744 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:06.201086 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:06.201099 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:06.201829 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:06.201833 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:06.201922 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:06.201951 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 21:12:06.202029 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 21:12:06.202106 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"mutex_wait_us":103,"thread_start_us":86,"threads_started":1}
I20260806 21:12:06.202845 28640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:06.197664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:46257 (local address 127.0.0.1:35400)
0806 21:12:06.198196 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:06.198212 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:06.198303 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:06.198728 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:06.198730 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:06.198745 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:12:06.198878 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:06.198886 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:06.199946 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:06.199949 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:06.201200 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:06.201209 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:06.202011 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:06.202015 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:06.202100 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:06.202123 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 21:12:06.202200 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 21:12:06.202265 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"mutex_wait_us":74,"thread_start_us":124,"threads_started":1}
I20260806 21:12:06.203286 28159 tablet_service.cc:1467] Processing CreateTablet for tablet 77145cca454a49baba503d618c56dba8 (DEFAULT_TABLE table=test-table [id=b6595813cc314db3b669c62ac3194d59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:12:06.203604 28159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77145cca454a49baba503d618c56dba8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:12:06.203578 27938 tablet_service.cc:1467] Processing CreateTablet for tablet 77145cca454a49baba503d618c56dba8 (DEFAULT_TABLE table=test-table [id=b6595813cc314db3b669c62ac3194d59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:12:06.203853 27938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77145cca454a49baba503d618c56dba8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:12:06.206090 28646 tablet_bootstrap.cc:492] T 77145cca454a49baba503d618c56dba8 P e8a457dc20ec498da1b35c91b59c632a: Bootstrap starting.
I20260806 21:12:06.206100 28641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:06.197887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53014 (local address 127.16.178.3:44489)
0806 21:12:06.198357 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:06.198361 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:06.198376 (+ 15us) server_negotiation.cc:407] Connection header received
0806 21:12:06.198432 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:06.198434 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:06.198499 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:06.198897 (+ 398us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:12:06.199013 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:06.199841 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:06.201121 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:06.201762 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:06.202756 (+ 994us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:06.202830 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:06.202866 (+ 36us) server_negotiation.cc:299] Negotiation successful
0806 21:12:06.202937 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":117,"threads_started":1}
I20260806 21:12:06.206605 28647 tablet_bootstrap.cc:492] T 77145cca454a49baba503d618c56dba8 P 97b7760ee07347c2990d40a6cbba0360: Bootstrap starting.
I20260806 21:12:06.206672 28381 tablet_service.cc:1467] Processing CreateTablet for tablet 77145cca454a49baba503d618c56dba8 (DEFAULT_TABLE table=test-table [id=b6595813cc314db3b669c62ac3194d59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:12:06.206945 28381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77145cca454a49baba503d618c56dba8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:12:06.207325 28646 tablet_bootstrap.cc:654] T 77145cca454a49baba503d618c56dba8 P e8a457dc20ec498da1b35c91b59c632a: Neither blocks nor log segments found. Creating new log.
I20260806 21:12:06.207671 28647 tablet_bootstrap.cc:654] T 77145cca454a49baba503d618c56dba8 P 97b7760ee07347c2990d40a6cbba0360: Neither blocks nor log segments found. Creating new log.
I20260806 21:12:06.208843 28646 tablet_bootstrap.cc:492] T 77145cca454a49baba503d618c56dba8 P e8a457dc20ec498da1b35c91b59c632a: No bootstrap required, opened a new log
I20260806 21:12:06.208931 28646 ts_tablet_manager.cc:1397] T 77145cca454a49baba503d618c56dba8 P e8a457dc20ec498da1b35c91b59c632a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 21:12:06.208992 28647 tablet_bootstrap.cc:492] T 77145cca454a49baba503d618c56dba8 P 97b7760ee07347c2990d40a6cbba0360: No bootstrap required, opened a new log
I20260806 21:12:06.209060 28647 ts_tablet_manager.cc:1397] T 77145cca454a49baba503d618c56dba8 P 97b7760ee07347c2990d40a6cbba0360: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 21:12:06.209399 28655 tablet_bootstrap.cc:492] T 77145cca454a49baba503d618c56dba8 P afab74cbf9344028ad587dc2c7896c5d: Bootstrap starting.
I20260806 21:12:06.209358 28646 raft_consensus.cc:348] T 77145cca454a49baba503d618c56dba8 P e8a457dc20ec498da1b35c91b59c632a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afab74cbf9344028ad587dc2c7896c5d" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44489 } } peers { permanent_uuid: "97b7760ee07347c2990d40a6cbba0360" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 46257 } } peers { permanent_uuid: "e8a457dc20ec498da1b35c91b59c632a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 41469 } }
I20260806 21:12:06.209481 28646 raft_consensus.cc:374] T 77145cca454a49baba503d618c56dba8 P e8a457dc20ec498da1b35c91b59c632a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:12:06.209548 28646 raft_consensus.cc:729] T 77145cca454a49baba503d618c56dba8 P e8a457dc20ec498da1b35c91b59c632a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8a457dc20ec498da1b35c91b59c632a, State: Initialized, Role: FOLLOWER
I20260806 21:12:06.209494 28647 raft_consensus.cc:348] T 77145cca454a49baba503d618c56dba8 P 97b7760ee07347c2990d40a6cbba0360 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afab74cbf9344028ad587dc2c7896c5d" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44489 } } peers { permanent_uuid: "97b7760ee07347c2990d40a6cbba0360" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 46257 } } peers { permanent_uuid: "e8a457dc20ec498da1b35c91b59c632a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 41469 } }
I20260806 21:12:06.209609 28647 raft_consensus.cc:374] T 77145cca454a49baba503d618c56dba8 P 97b7760ee07347c2990d40a6cbba0360 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:12:06.209653 28647 raft_consensus.cc:729] T 77145cca454a49baba503d618c56dba8 P 97b7760ee07347c2990d40a6cbba0360 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97b7760ee07347c2990d40a6cbba0360, State: Initialized, Role: FOLLOWER
I20260806 21:12:06.209708 28646 consensus_queue.cc:260] T 77145cca454a49baba503d618c56dba8 P e8a457dc20ec498da1b35c91b59c632a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afab74cbf9344028ad587dc2c7896c5d" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44489 } } peers { permanent_uuid: "97b7760ee07347c2990d40a6cbba0360" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 46257 } } peers { permanent_uuid: "e8a457dc20ec498da1b35c91b59c632a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 41469 } }
I20260806 21:12:06.209800 28647 consensus_queue.cc:260] T 77145cca454a49baba503d618c56dba8 P 97b7760ee07347c2990d40a6cbba0360 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afab74cbf9344028ad587dc2c7896c5d" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44489 } } peers { permanent_uuid: "97b7760ee07347c2990d40a6cbba0360" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 46257 } } peers { permanent_uuid: "e8a457dc20ec498da1b35c91b59c632a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 41469 } }
I20260806 21:12:06.210067 28646 ts_tablet_manager.cc:1428] T 77145cca454a49baba503d618c56dba8 P e8a457dc20ec498da1b35c91b59c632a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 21:12:06.210178 28647 ts_tablet_manager.cc:1428] T 77145cca454a49baba503d618c56dba8 P 97b7760ee07347c2990d40a6cbba0360: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 21:12:06.210357 28655 tablet_bootstrap.cc:654] T 77145cca454a49baba503d618c56dba8 P afab74cbf9344028ad587dc2c7896c5d: Neither blocks nor log segments found. Creating new log.
I20260806 21:12:06.210919 28623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:06.210454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44091 (local address 127.16.178.2:55723)
0806 21:12:06.210781 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:06.210842 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240}
I20260806 21:12:06.211766 28623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:06.211402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44091 (local address 127.16.178.2:41433)
0806 21:12:06.211636 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:06.211692 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171}
I20260806 21:12:06.212126 28624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:06.211582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44091 (local address 127.16.178.1:47309)
0806 21:12:06.211980 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:06.212037 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323}
I20260806 21:12:06.213030 28263 heartbeater.cc:503] Master 127.16.178.60:46781 was elected leader, sending a full tablet report...
I20260806 21:12:06.213402 28045 heartbeater.cc:503] Master 127.16.178.60:46781 was elected leader, sending a full tablet report...
I20260806 21:12:06.214649 28624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:06.214185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44091 (local address 127.16.178.1:45215)
0806 21:12:06.214511 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:06.214575 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199}
I20260806 21:12:06.215646 28655 tablet_bootstrap.cc:492] T 77145cca454a49baba503d618c56dba8 P afab74cbf9344028ad587dc2c7896c5d: No bootstrap required, opened a new log
I20260806 21:12:06.215759 28655 ts_tablet_manager.cc:1397] T 77145cca454a49baba503d618c56dba8 P afab74cbf9344028ad587dc2c7896c5d: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260806 21:12:06.216266 28655 raft_consensus.cc:348] T 77145cca454a49baba503d618c56dba8 P afab74cbf9344028ad587dc2c7896c5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afab74cbf9344028ad587dc2c7896c5d" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44489 } } peers { permanent_uuid: "97b7760ee07347c2990d40a6cbba0360" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 46257 } } peers { permanent_uuid: "e8a457dc20ec498da1b35c91b59c632a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 41469 } }
I20260806 21:12:06.216385 28655 raft_consensus.cc:374] T 77145cca454a49baba503d618c56dba8 P afab74cbf9344028ad587dc2c7896c5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:12:06.216420 28655 raft_consensus.cc:729] T 77145cca454a49baba503d618c56dba8 P afab74cbf9344028ad587dc2c7896c5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afab74cbf9344028ad587dc2c7896c5d, State: Initialized, Role: FOLLOWER
I20260806 21:12:06.216565 28655 consensus_queue.cc:260] T 77145cca454a49baba503d618c56dba8 P afab74cbf9344028ad587dc2c7896c5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afab74cbf9344028ad587dc2c7896c5d" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44489 } } peers { permanent_uuid: "97b7760ee07347c2990d40a6cbba0360" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 46257 } } peers { permanent_uuid: "e8a457dc20ec498da1b35c91b59c632a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 41469 } }
I20260806 21:12:06.216957 28655 ts_tablet_manager.cc:1428] T 77145cca454a49baba503d618c56dba8 P afab74cbf9344028ad587dc2c7896c5d: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 21:12:06.217557 28620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:06.217295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44091 (local address 127.16.178.3:52995)
0806 21:12:06.217414 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:06.217476 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 21:12:06.218262 28620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:06.218007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44091 (local address 127.16.178.3:39737)
0806 21:12:06.218140 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:06.218198 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 21:12:06.218787 28490 heartbeater.cc:503] Master 127.16.178.60:46781 was elected leader, sending a full tablet report...
I20260806 21:12:06.231278 28657 raft_consensus.cc:482] T 77145cca454a49baba503d618c56dba8 P 97b7760ee07347c2990d40a6cbba0360 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:12:06.231452 28657 raft_consensus.cc:504] T 77145cca454a49baba503d618c56dba8 P 97b7760ee07347c2990d40a6cbba0360 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afab74cbf9344028ad587dc2c7896c5d" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44489 } } peers { permanent_uuid: "97b7760ee07347c2990d40a6cbba0360" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 46257 } } peers { permanent_uuid: "e8a457dc20ec498da1b35c91b59c632a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 41469 } }
I20260806 21:12:06.232833 28657 leader_election.cc:288] T 77145cca454a49baba503d618c56dba8 P 97b7760ee07347c2990d40a6cbba0360 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers afab74cbf9344028ad587dc2c7896c5d (127.16.178.3:44489), e8a457dc20ec498da1b35c91b59c632a (127.16.178.2:41469)
I20260806 21:12:06.237883 28641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:06.233185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:39801 (local address 127.16.178.3:44489)
0806 21:12:06.233330 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:06.233335 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:06.233352 (+ 17us) server_negotiation.cc:407] Connection header received
0806 21:12:06.233508 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:06.233510 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:06.233559 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:06.233620 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:12:06.234000 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:06.234765 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:06.236774 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:06.237234 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:06.237359 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:06.237421 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:06.237674 (+ 253us) server_negotiation.cc:299] Negotiation successful
0806 21:12:06.237755 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 21:12:06.237883 28624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:06.233096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.3:44489 (local address 127.16.178.1:39801)
0806 21:12:06.233242 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:06.233258 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:06.233377 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:06.233697 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:06.233700 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:06.233725 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:12:06.233880 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:06.233887 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:06.234922 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:06.234926 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:06.236573 (+ 1647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:06.236590 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:06.237366 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:06.237370 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:06.237528 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:06.237565 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 21:12:06.237634 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 21:12:06.237724 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":71}
I20260806 21:12:06.237902 28643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:06.233225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:33951 (local address 127.16.178.2:41469)
0806 21:12:06.233344 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:06.233349 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:06.233777 (+ 428us) server_negotiation.cc:407] Connection header received
0806 21:12:06.233871 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:06.233874 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:06.233915 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:06.233987 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:12:06.234196 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:06.234969 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:06.236760 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:06.237240 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:06.237370 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:06.237428 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:06.237696 (+ 268us) server_negotiation.cc:299] Negotiation successful
0806 21:12:06.237779 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 21:12:06.237912 28667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:06.233105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.2:41469 (local address 127.16.178.1:33951)
0806 21:12:06.233673 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:06.233689 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:06.233789 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:06.234004 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:06.234006 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:06.234022 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:12:06.234103 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:06.234108 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:06.235088 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:06.235091 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:06.236574 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:06.236590 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:06.237378 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:06.237382 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:06.237568 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:06.237589 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 21:12:06.237662 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 21:12:06.237746 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"mutex_wait_us":270,"thread_start_us":145,"threads_started":1}
I20260806 21:12:06.238704 28410 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77145cca454a49baba503d618c56dba8" candidate_uuid: "97b7760ee07347c2990d40a6cbba0360" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afab74cbf9344028ad587dc2c7896c5d" is_pre_election: true
I20260806 21:12:06.238749 28199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77145cca454a49baba503d618c56dba8" candidate_uuid: "97b7760ee07347c2990d40a6cbba0360" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8a457dc20ec498da1b35c91b59c632a" is_pre_election: true
I20260806 21:12:06.238899 28410 raft_consensus.cc:2454] T 77145cca454a49baba503d618c56dba8 P afab74cbf9344028ad587dc2c7896c5d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97b7760ee07347c2990d40a6cbba0360 in term 0.
I20260806 21:12:06.239028 28199 raft_consensus.cc:2454] T 77145cca454a49baba503d618c56dba8 P e8a457dc20ec498da1b35c91b59c632a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97b7760ee07347c2990d40a6cbba0360 in term 0.
I20260806 21:12:06.239372 27864 leader_election.cc:302] T 77145cca454a49baba503d618c56dba8 P 97b7760ee07347c2990d40a6cbba0360 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 97b7760ee07347c2990d40a6cbba0360, afab74cbf9344028ad587dc2c7896c5d, e8a457dc20ec498da1b35c91b59c632a; no voters:
I20260806 21:12:06.239710 28657 raft_consensus.cc:2781] T 77145cca454a49baba503d618c56dba8 P 97b7760ee07347c2990d40a6cbba0360 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:12:06.239779 28657 raft_consensus.cc:482] T 77145cca454a49baba503d618c56dba8 P 97b7760ee07347c2990d40a6cbba0360 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:12:06.239881 28657 raft_consensus.cc:3037] T 77145cca454a49baba503d618c56dba8 P 97b7760ee07347c2990d40a6cbba0360 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:12:06.241221 28657 raft_consensus.cc:504] T 77145cca454a49baba503d618c56dba8 P 97b7760ee07347c2990d40a6cbba0360 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afab74cbf9344028ad587dc2c7896c5d" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44489 } } peers { permanent_uuid: "97b7760ee07347c2990d40a6cbba0360" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 46257 } } peers { permanent_uuid: "e8a457dc20ec498da1b35c91b59c632a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 41469 } }
I20260806 21:12:06.241642 28657 leader_election.cc:288] T 77145cca454a49baba503d618c56dba8 P 97b7760ee07347c2990d40a6cbba0360 [CANDIDATE]: Term 1 election: Requested vote from peers afab74cbf9344028ad587dc2c7896c5d (127.16.178.3:44489), e8a457dc20ec498da1b35c91b59c632a (127.16.178.2:41469)
I20260806 21:12:06.242039 28199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77145cca454a49baba503d618c56dba8" candidate_uuid: "97b7760ee07347c2990d40a6cbba0360" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8a457dc20ec498da1b35c91b59c632a"
I20260806 21:12:06.242187 28199 raft_consensus.cc:3037] T 77145cca454a49baba503d618c56dba8 P e8a457dc20ec498da1b35c91b59c632a [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:12:06.242216 28410 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77145cca454a49baba503d618c56dba8" candidate_uuid: "97b7760ee07347c2990d40a6cbba0360" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afab74cbf9344028ad587dc2c7896c5d"
I20260806 21:12:06.242362 28410 raft_consensus.cc:3037] T 77145cca454a49baba503d618c56dba8 P afab74cbf9344028ad587dc2c7896c5d [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:12:06.243292 28199 raft_consensus.cc:2454] T 77145cca454a49baba503d618c56dba8 P e8a457dc20ec498da1b35c91b59c632a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97b7760ee07347c2990d40a6cbba0360 in term 1.
I20260806 21:12:06.243431 28410 raft_consensus.cc:2454] T 77145cca454a49baba503d618c56dba8 P afab74cbf9344028ad587dc2c7896c5d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97b7760ee07347c2990d40a6cbba0360 in term 1.
I20260806 21:12:06.243644 27865 leader_election.cc:302] T 77145cca454a49baba503d618c56dba8 P 97b7760ee07347c2990d40a6cbba0360 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97b7760ee07347c2990d40a6cbba0360, e8a457dc20ec498da1b35c91b59c632a; no voters:
I20260806 21:12:06.243862 28657 raft_consensus.cc:2781] T 77145cca454a49baba503d618c56dba8 P 97b7760ee07347c2990d40a6cbba0360 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:12:06.244149 28657 raft_consensus.cc:686] T 77145cca454a49baba503d618c56dba8 P 97b7760ee07347c2990d40a6cbba0360 [term 1 LEADER]: Becoming Leader. State: Replica: 97b7760ee07347c2990d40a6cbba0360, State: Running, Role: LEADER
I20260806 21:12:06.244617 28657 consensus_queue.cc:237] T 77145cca454a49baba503d618c56dba8 P 97b7760ee07347c2990d40a6cbba0360 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afab74cbf9344028ad587dc2c7896c5d" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44489 } } peers { permanent_uuid: "97b7760ee07347c2990d40a6cbba0360" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 46257 } } peers { permanent_uuid: "e8a457dc20ec498da1b35c91b59c632a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 41469 } }
I20260806 21:12:06.244992 28667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:06.244671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44091 (local address 127.16.178.1:50909)
0806 21:12:06.244847 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:06.244911 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260806 21:12:06.247223 27760 catalog_manager.cc:5495] T 77145cca454a49baba503d618c56dba8 P 97b7760ee07347c2990d40a6cbba0360 reported cstate change: term changed from 0 to 1, leader changed from <none> to 97b7760ee07347c2990d40a6cbba0360 (127.16.178.1). New cstate: current_term: 1 leader_uuid: "97b7760ee07347c2990d40a6cbba0360" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "afab74cbf9344028ad587dc2c7896c5d" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97b7760ee07347c2990d40a6cbba0360" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 46257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8a457dc20ec498da1b35c91b59c632a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 41469 } health_report { overall_health: UNKNOWN } } }
I20260806 21:12:06.268504 28644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:06.264496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35416 (local address 127.16.178.1:46257)
0806 21:12:06.264663 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:06.264667 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:06.264715 (+ 48us) server_negotiation.cc:407] Connection header received
0806 21:12:06.264801 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:06.264803 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:06.264860 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:06.264945 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:12:06.265172 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:06.265671 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:06.266893 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:06.267118 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:06.267202 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:06.267496 (+ 294us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:12:06.268051 (+ 555us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:12:06.268181 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:06.268269 (+ 88us) server_negotiation.cc:299] Negotiation successful
0806 21:12:06.268331 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 21:12:06.271507 28410 raft_consensus.cc:1261] T 77145cca454a49baba503d618c56dba8 P afab74cbf9344028ad587dc2c7896c5d [term 1 FOLLOWER]: Refusing update from remote peer 97b7760ee07347c2990d40a6cbba0360: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:12:06.271796 28199 raft_consensus.cc:1261] T 77145cca454a49baba503d618c56dba8 P e8a457dc20ec498da1b35c91b59c632a [term 1 FOLLOWER]: Refusing update from remote peer 97b7760ee07347c2990d40a6cbba0360: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:12:06.272100 28657 consensus_queue.cc:1035] T 77145cca454a49baba503d618c56dba8 P 97b7760ee07347c2990d40a6cbba0360 [LEADER]: Connected to new peer: Peer: permanent_uuid: "afab74cbf9344028ad587dc2c7896c5d" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:12:06.272367 28668 consensus_queue.cc:1035] T 77145cca454a49baba503d618c56dba8 P 97b7760ee07347c2990d40a6cbba0360 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8a457dc20ec498da1b35c91b59c632a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 41469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:12:06.274072 28620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:06.273741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44091 (local address 127.16.178.3:47399)
0806 21:12:06.273925 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:06.273989 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260806 21:12:06.275581 28623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:06.275310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44091 (local address 127.16.178.2:40301)
0806 21:12:06.275443 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:06.275504 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 21:12:06.277172 28673 mvcc.cc:205] Tried to move back new op lower bound from 7315663774802624512 to 7315663774700990464. Current Snapshot: MvccSnapshot[applied={T|T < 7315663774802624512}]
I20260806 21:12:06.278177 28667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:06.277679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44091 (local address 127.16.178.1:44949)
0806 21:12:06.278044 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:06.278104 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":291}
I20260806 21:12:06.279136 28620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:06.278764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44091 (local address 127.16.178.3:57733)
0806 21:12:06.279016 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:06.279068 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187}
I20260806 21:12:06.279687 28667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:06.279325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44091 (local address 127.16.178.1:50065)
0806 21:12:06.279570 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:06.279623 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182}
I20260806 21:12:06.282258 28623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:06.281206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44091 (local address 127.16.178.2:53289)
0806 21:12:06.282117 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:06.282175 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":833}
I20260806 21:12:06.287422 27760 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54854:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:12:06.287598 27760 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54854:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:12:06.291724 27760 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2119692719ab4361bbbca6732e4d197a: Sending DeleteTablet for 3 replicas of tablet 77145cca454a49baba503d618c56dba8
I20260806 21:12:06.292573 28381 tablet_service.cc:1514] Processing DeleteTablet for tablet 77145cca454a49baba503d618c56dba8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:12:06 UTC) from {username='oryx', principal='oryx/127.16.178.60@KRBTEST.COM'} at 127.0.0.1:53014
I20260806 21:12:06.292727 27938 tablet_service.cc:1514] Processing DeleteTablet for tablet 77145cca454a49baba503d618c56dba8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:12:06 UTC) from {username='oryx', principal='oryx/127.16.178.60@KRBTEST.COM'} at 127.0.0.1:35400
I20260806 21:12:06.292762 28159 tablet_service.cc:1514] Processing DeleteTablet for tablet 77145cca454a49baba503d618c56dba8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:12:06 UTC) from {username='oryx', principal='oryx/127.16.178.60@KRBTEST.COM'} at 127.0.0.1:58956
I20260806 21:12:06.293160 28692 tablet_replica.cc:330] T 77145cca454a49baba503d618c56dba8 P e8a457dc20ec498da1b35c91b59c632a: stopping tablet replica
I20260806 21:12:06.293262 28687 tablet_replica.cc:330] T 77145cca454a49baba503d618c56dba8 P afab74cbf9344028ad587dc2c7896c5d: stopping tablet replica
I20260806 21:12:06.293298 28692 raft_consensus.cc:2229] T 77145cca454a49baba503d618c56dba8 P e8a457dc20ec498da1b35c91b59c632a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:12:06.293380 28687 raft_consensus.cc:2229] T 77145cca454a49baba503d618c56dba8 P afab74cbf9344028ad587dc2c7896c5d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:12:06.293438 28692 raft_consensus.cc:2258] T 77145cca454a49baba503d618c56dba8 P e8a457dc20ec498da1b35c91b59c632a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:12:06.293509 28687 raft_consensus.cc:2258] T 77145cca454a49baba503d618c56dba8 P afab74cbf9344028ad587dc2c7896c5d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:12:06.294070 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 27703
I20260806 21:12:06.294250 28687 ts_tablet_manager.cc:1905] T 77145cca454a49baba503d618c56dba8 P afab74cbf9344028ad587dc2c7896c5d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:12:06.294251 28692 ts_tablet_manager.cc:1905] T 77145cca454a49baba503d618c56dba8 P e8a457dc20ec498da1b35c91b59c632a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:12:06.296669 28687 ts_tablet_manager.cc:1918] T 77145cca454a49baba503d618c56dba8 P afab74cbf9344028ad587dc2c7896c5d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:12:06.296731 28687 log.cc:1196] T 77145cca454a49baba503d618c56dba8 P afab74cbf9344028ad587dc2c7896c5d: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-2/wal/wals/77145cca454a49baba503d618c56dba8
I20260806 21:12:06.296942 28687 ts_tablet_manager.cc:1939] T 77145cca454a49baba503d618c56dba8 P afab74cbf9344028ad587dc2c7896c5d: Deleting consensus metadata
I20260806 21:12:06.297380 28692 ts_tablet_manager.cc:1918] T 77145cca454a49baba503d618c56dba8 P e8a457dc20ec498da1b35c91b59c632a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:12:06.297454 28692 log.cc:1196] T 77145cca454a49baba503d618c56dba8 P e8a457dc20ec498da1b35c91b59c632a: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786050686797304-17096-0/minicluster-data/ts-1/wal/wals/77145cca454a49baba503d618c56dba8
I20260806 21:12:06.297741 28692 ts_tablet_manager.cc:1939] T 77145cca454a49baba503d618c56dba8 P e8a457dc20ec498da1b35c91b59c632a: Deleting consensus metadata
I20260806 21:12:06.298448 27740 catalog_manager.cc:4841] TS e8a457dc20ec498da1b35c91b59c632a (127.16.178.2:41469): tablet 77145cca454a49baba503d618c56dba8 (table test-table [id=b6595813cc314db3b669c62ac3194d59]) successfully deleted
I20260806 21:12:06.300884 27739 catalog_manager.cc:4841] TS afab74cbf9344028ad587dc2c7896c5d (127.16.178.3:44489): tablet 77145cca454a49baba503d618c56dba8 (table test-table [id=b6595813cc314db3b669c62ac3194d59]) successfully deleted
I20260806 21:12:06.307928 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 28058
I20260806 21:12:06.308015 28640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:06.307730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:46257 (local address 127.0.0.1:35422)
0806 21:12:06.307876 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:06.307928 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.1:46257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 21:12:06.315444 27739 catalog_manager.cc:4584] TS 97b7760ee07347c2990d40a6cbba0360 (127.16.178.1:46257): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 77145cca454a49baba503d618c56dba8: Network error: Client connection negotiation failed: client connection to 127.16.178.1:46257: connect: Connection refused (error 111)
I20260806 21:12:06.323491 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 28276
I20260806 21:12:06.336437 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 27263
I20260806 21:12:06.354045 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 27415
2026-08-06T21:12:06Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7185 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:12:06 dist-test-slave-lgr5 krb5kdc[28695](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:12:06 dist-test-slave-lgr5 krb5kdc[28695](info): set up 2 sockets
Aug 06 21:12:06 dist-test-slave-lgr5 krb5kdc[28695](info): commencing operation
krb5kdc: starting...
W20260806 21:12:07.514070 17096 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Aug 06 21:12:07 dist-test-slave-lgr5 krb5kdc[28695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050727, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:12:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:12:07Z Disabled control of system clock
WARNING: no policy specified for oryx/127.16.178.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.178.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.178.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.62.keytab.
Entry for principal oryx/127.16.178.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.62.keytab.
I20260806 21:12:07.643452 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:46619
--webserver_interface=127.16.178.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.62.keytab
--principal=oryx/127.16.178.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:35595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:46619
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:12:07.761366 28712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:12:07.761675 28712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:12:07.761736 28712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:12:07.765842 28712 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:12:07.765908 28712 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 21:12:07.765930 28712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:12:07.765955 28712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:12:07.765980 28712 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:12:07.766006 28712 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:12:07.771453 28712 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:35595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:46619
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.62.keytab
--principal=oryx/127.16.178.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:46619
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.178.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:12:07.772728 28712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:12:07.773839 28712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:12:07.782707 28717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:07.782779 28719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:07.782709 28725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:07.782967 28712 server_base.cc:1034] running on GCE node
I20260806 21:12:07.783727 28712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:12:07.785001 28712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:12:07.786182 28712 hybrid_clock.cc:648] HybridClock initialized: now 1786050727786162 us; error 35 us; skew 500 ppm
Aug 06 21:12:07 dist-test-slave-lgr5 krb5kdc[28695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050727, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.178.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:12:07.789181 28712 init.cc:377] Logged in from keytab as oryx/127.16.178.62@KRBTEST.COM (short username oryx)
I20260806 21:12:07.790131 28712 webserver.cc:460] Webserver started at http://127.16.178.62:41177/ using document root <none> and password file <none>
I20260806 21:12:07.790697 28712 fs_manager.cc:359] Metadata directory not provided
I20260806 21:12:07.790784 28712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:12:07.791018 28712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:12:07.793231 28712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "9f793d6ef89b4cd0a78f73c7c77df3f9"
format_stamp: "Formatted at 2026-08-06 21:12:07 on dist-test-slave-lgr5"
server_key: "9284407fccd3f72c2fb7d45bca4f7e57"
server_key_iv: "f4c2844cf107907a826ff1181b6142dd"
server_key_version: "encryptionkey@0"
I20260806 21:12:07.793844 28712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "9f793d6ef89b4cd0a78f73c7c77df3f9"
format_stamp: "Formatted at 2026-08-06 21:12:07 on dist-test-slave-lgr5"
server_key: "9284407fccd3f72c2fb7d45bca4f7e57"
server_key_iv: "f4c2844cf107907a826ff1181b6142dd"
server_key_version: "encryptionkey@0"
I20260806 21:12:07.797837 28712 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:12:07.800469 28740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:07.801697 28712 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 21:12:07.801834 28712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "9f793d6ef89b4cd0a78f73c7c77df3f9"
format_stamp: "Formatted at 2026-08-06 21:12:07 on dist-test-slave-lgr5"
server_key: "9284407fccd3f72c2fb7d45bca4f7e57"
server_key_iv: "f4c2844cf107907a826ff1181b6142dd"
server_key_version: "encryptionkey@0"
I20260806 21:12:07.801942 28712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:12:07.815128 28712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:12:07.818303 28712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:12:07.818497 28712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:12:07.826198 28712 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.62:46619
I20260806 21:12:07.826236 28833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.62:46619 every 8 connection(s)
I20260806 21:12:07.827180 28712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:12:07.830113 28834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:12:07.831144 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 28712
I20260806 21:12:07.831275 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:12:07.831561 17096 external_mini_cluster.cc:1428] Setting key b8ae6a55e6f9dd06059dfe71e065547d
I20260806 21:12:07.836416 28834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f793d6ef89b4cd0a78f73c7c77df3f9: Bootstrap starting.
I20260806 21:12:07.839138 28834 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f793d6ef89b4cd0a78f73c7c77df3f9: Neither blocks nor log segments found. Creating new log.
I20260806 21:12:07.840097 28834 log.cc:824] T 00000000000000000000000000000000 P 9f793d6ef89b4cd0a78f73c7c77df3f9: Log is configured to *not* fsync() on all Append() calls
Aug 06 21:12:07 dist-test-slave-lgr5 krb5kdc[28695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050727, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM
I20260806 21:12:07.842548 28834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f793d6ef89b4cd0a78f73c7c77df3f9: No bootstrap required, opened a new log
I20260806 21:12:07.846580 28834 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f793d6ef89b4cd0a78f73c7c77df3f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f793d6ef89b4cd0a78f73c7c77df3f9" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 46619 } }
I20260806 21:12:07.846851 28834 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f793d6ef89b4cd0a78f73c7c77df3f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:12:07.846909 28834 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f793d6ef89b4cd0a78f73c7c77df3f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f793d6ef89b4cd0a78f73c7c77df3f9, State: Initialized, Role: FOLLOWER
I20260806 21:12:07.846971 28837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:07.832893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39252 (local address 127.16.178.62:46619)
0806 21:12:07.833279 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:07.833295 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:07.833334 (+ 39us) server_negotiation.cc:407] Connection header received
0806 21:12:07.833951 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:07.833967 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:07.834289 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:07.834556 (+ 267us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:07.835490 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:07.836522 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:07.837651 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:07.838131 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:07.838261 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:07.841105 (+ 2844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:07.841111 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:07.841127 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:07.841157 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:07.843885 (+ 2728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:07.844362 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:07.844369 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:07.844376 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:07.844518 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:07.844755 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:07.844758 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:07.844760 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:07.845274 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:07.845474 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:07.845678 (+ 204us) server_negotiation.cc:299] Negotiation successful
0806 21:12:07.846000 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":103,"threads_started":1}
I20260806 21:12:07.847373 28834 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f793d6ef89b4cd0a78f73c7c77df3f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f793d6ef89b4cd0a78f73c7c77df3f9" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 46619 } }
I20260806 21:12:07.847527 28834 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f793d6ef89b4cd0a78f73c7c77df3f9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:12:07.847595 28834 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f793d6ef89b4cd0a78f73c7c77df3f9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:12:07.847692 28834 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f793d6ef89b4cd0a78f73c7c77df3f9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:12:07.848966 28834 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f793d6ef89b4cd0a78f73c7c77df3f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f793d6ef89b4cd0a78f73c7c77df3f9" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 46619 } }
I20260806 21:12:07.849347 28834 leader_election.cc:302] T 00000000000000000000000000000000 P 9f793d6ef89b4cd0a78f73c7c77df3f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f793d6ef89b4cd0a78f73c7c77df3f9; no voters:
I20260806 21:12:07.849735 28834 leader_election.cc:288] T 00000000000000000000000000000000 P 9f793d6ef89b4cd0a78f73c7c77df3f9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:12:07.849838 28840 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f793d6ef89b4cd0a78f73c7c77df3f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:12:07.850056 28840 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f793d6ef89b4cd0a78f73c7c77df3f9 [term 1 LEADER]: Becoming Leader. State: Replica: 9f793d6ef89b4cd0a78f73c7c77df3f9, State: Running, Role: LEADER
I20260806 21:12:07.850395 28840 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f793d6ef89b4cd0a78f73c7c77df3f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f793d6ef89b4cd0a78f73c7c77df3f9" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 46619 } }
I20260806 21:12:07.850960 28834 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f793d6ef89b4cd0a78f73c7c77df3f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:12:07.852377 28841 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f793d6ef89b4cd0a78f73c7c77df3f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f793d6ef89b4cd0a78f73c7c77df3f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f793d6ef89b4cd0a78f73c7c77df3f9" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 46619 } } }
I20260806 21:12:07.852401 28842 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f793d6ef89b4cd0a78f73c7c77df3f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f793d6ef89b4cd0a78f73c7c77df3f9. Latest consensus state: current_term: 1 leader_uuid: "9f793d6ef89b4cd0a78f73c7c77df3f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f793d6ef89b4cd0a78f73c7c77df3f9" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 46619 } } }
I20260806 21:12:07.852526 28842 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f793d6ef89b4cd0a78f73c7c77df3f9 [sys.catalog]: This master's current role is: LEADER
I20260806 21:12:07.852524 28841 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f793d6ef89b4cd0a78f73c7c77df3f9 [sys.catalog]: This master's current role is: LEADER
I20260806 21:12:07.853056 28856 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:12:07.855953 28856 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:12:07.861581 28856 catalog_manager.cc:1347] Generated new cluster ID: 7bb491dac6214158a5cd04b927cf5c86
I20260806 21:12:07.861673 28856 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:12:07.875756 28856 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:12:07.876843 28856 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:12:07.881531 28856 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f793d6ef89b4cd0a78f73c7c77df3f9: Generated new TSK 0
I20260806 21:12:07.882239 28856 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.16.178.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.178.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.178.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.1.keytab.
Entry for principal oryx/127.16.178.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.1.keytab.
I20260806 21:12:07.917259 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.1:0
--local_ip_for_outbound_sockets=127.16.178.1
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:46619
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.1.keytab
--principal=oryx/127.16.178.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:35595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:12:08.036795 28870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:12:08.037137 28870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:12:08.037204 28870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:12:08.041523 28870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:12:08.041608 28870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:12:08.041666 28870 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 21:12:08.041726 28870 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.1
I20260806 21:12:08.047544 28870 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:35595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.1.keytab
--principal=oryx/127.16.178.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.178.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.178.62:46619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.1
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:12:08.048847 28870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:12:08.049959 28870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:12:08.059911 28880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:08.059911 28878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:08.059906 28885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:08.060102 28870 server_base.cc:1034] running on GCE node
I20260806 21:12:08.060683 28870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:12:08.061323 28870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:12:08.062482 28870 hybrid_clock.cc:648] HybridClock initialized: now 1786050728062467 us; error 30 us; skew 500 ppm
Aug 06 21:12:08 dist-test-slave-lgr5 krb5kdc[28695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050728, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.178.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:12:08.065315 28870 init.cc:377] Logged in from keytab as oryx/127.16.178.1@KRBTEST.COM (short username oryx)
I20260806 21:12:08.066207 28870 webserver.cc:460] Webserver started at http://127.16.178.1:35071/ using document root <none> and password file <none>
I20260806 21:12:08.066754 28870 fs_manager.cc:359] Metadata directory not provided
I20260806 21:12:08.066820 28870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:12:08.067028 28870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:12:08.069099 28870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-0/data/instance:
uuid: "b45a1e0cc11d4f9c9b52563a245e291b"
format_stamp: "Formatted at 2026-08-06 21:12:08 on dist-test-slave-lgr5"
server_key: "824b0adf6b1eed2f21e2d1f8191e6fdf"
server_key_iv: "bd7302093814a6982263547f063bfcfb"
server_key_version: "encryptionkey@0"
I20260806 21:12:08.069630 28870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance:
uuid: "b45a1e0cc11d4f9c9b52563a245e291b"
format_stamp: "Formatted at 2026-08-06 21:12:08 on dist-test-slave-lgr5"
server_key: "824b0adf6b1eed2f21e2d1f8191e6fdf"
server_key_iv: "bd7302093814a6982263547f063bfcfb"
server_key_version: "encryptionkey@0"
I20260806 21:12:08.073532 28870 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:12:08.075995 28895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:08.077309 28870 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 21:12:08.077448 28870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-0/wal
uuid: "b45a1e0cc11d4f9c9b52563a245e291b"
format_stamp: "Formatted at 2026-08-06 21:12:08 on dist-test-slave-lgr5"
server_key: "824b0adf6b1eed2f21e2d1f8191e6fdf"
server_key_iv: "bd7302093814a6982263547f063bfcfb"
server_key_version: "encryptionkey@0"
I20260806 21:12:08.077562 28870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:12:08.097746 28870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:12:08.101205 28870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:12:08.101401 28870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:12:08.102936 28870 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:12:08.102989 28870 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:08.103045 28870 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:12:08.103081 28870 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 21:12:08 dist-test-slave-lgr5 krb5kdc[28695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050728, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.178.1@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM
I20260806 21:12:08.113497 28870 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.1:32881
I20260806 21:12:08.113704 29072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.1:32881 every 8 connection(s)
I20260806 21:12:08.114522 28870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
I20260806 21:12:08.114713 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 28870
I20260806 21:12:08.114847 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance
I20260806 21:12:08.115199 17096 external_mini_cluster.cc:1428] Setting key a86120f54134c7050bc8fbd2333445f5
I20260806 21:12:08.117082 28837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:08.103875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:50783 (local address 127.16.178.62:46619)
0806 21:12:08.104004 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:08.104007 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:08.105142 (+ 1135us) server_negotiation.cc:407] Connection header received
0806 21:12:08.106396 (+ 1254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:08.106400 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:08.106470 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:08.106555 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:08.107943 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.108612 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.109892 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.110134 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.110230 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:08.112930 (+ 2700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:08.112935 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:08.112938 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:08.112969 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:08.114688 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:08.115312 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:08.115316 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:08.115318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:08.115420 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:08.115780 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:08.115784 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:08.115785 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:08.116016 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:08.116143 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:08.116678 (+ 535us) server_negotiation.cc:299] Negotiation successful
0806 21:12:08.116922 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 21:12:08.117723 28908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:08.104442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:46619 (local address 127.16.178.1:50783)
0806 21:12:08.104976 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:08.105038 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:08.106123 (+ 1085us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:08.106818 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:08.106828 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:08.107170 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:08.107786 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:08.107802 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.108752 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.108756 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:08.109763 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:08.109787 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.110317 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.110323 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:08.110495 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:08.111072 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:08.111090 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:08.112742 (+ 1652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:12:08.114812 (+ 2070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:08.114819 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:08.114833 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:08.115188 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:08.115525 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:08.115528 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:08.115530 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:08.115694 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:08.116146 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:12:08.116153 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:12:08.116422 (+ 269us) client_negotiation.cc:769] Sending connection context
0806 21:12:08.116611 (+ 189us) client_negotiation.cc:240] Negotiation successful
0806 21:12:08.116879 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":144,"threads_started":1}
I20260806 21:12:08.119055 29073 heartbeater.cc:347] Connected to a master server at 127.16.178.62:46619
I20260806 21:12:08.119215 29073 heartbeater.cc:464] Registering TS with master...
I20260806 21:12:08.119720 29073 heartbeater.cc:511] Master 127.16.178.62:46619 requested a full tablet report, sending...
I20260806 21:12:08.121359 28763 ts_manager.cc:194] Registered new tserver with Master: b45a1e0cc11d4f9c9b52563a245e291b (127.16.178.1:32881)
I20260806 21:12:08.122622 28763 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.178.1@KRBTEST.COM'} at 127.16.178.1:50783
WARNING: no policy specified for oryx/127.16.178.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.178.2@KRBTEST.COM" created.
I20260806 21:12:08.130684 29091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:08.121514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:46619 (local address 127.16.178.1:36239)
0806 21:12:08.121841 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:08.121856 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:08.121945 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:08.123491 (+ 1546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:08.123495 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:08.123718 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:08.123870 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:08.123879 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.124937 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.124941 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:08.125828 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:08.125840 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.126344 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.126348 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:08.126427 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:08.126901 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:08.126911 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:08.127247 (+ 336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:12:08.129114 (+ 1867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:08.129118 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:08.129120 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:08.129399 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:08.129703 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:08.129705 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:08.129706 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:08.129775 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:08.130207 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:12:08.130210 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:12:08.130297 (+ 87us) client_negotiation.cc:769] Sending connection context
0806 21:12:08.130405 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 21:12:08.130530 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"mutex_wait_us":2,"thread_start_us":119,"threads_started":1}
I20260806 21:12:08.130787 28837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:08.121441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:36239 (local address 127.16.178.62:46619)
0806 21:12:08.123251 (+ 1810us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:08.123255 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:08.123268 (+ 13us) server_negotiation.cc:407] Connection header received
0806 21:12:08.123303 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:08.123306 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:08.123360 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:08.123442 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:08.124016 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.124791 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.125958 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.126218 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.126301 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:08.127403 (+ 1102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:08.127407 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:08.127409 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:08.127443 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:08.128979 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:08.129503 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:08.129507 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:08.129509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:08.129613 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:08.129852 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:08.129855 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:08.129856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:08.130099 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:08.130222 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:08.130483 (+ 261us) server_negotiation.cc:299] Negotiation successful
0806 21:12:08.130613 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1731}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.178.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.2.keytab.
Entry for principal oryx/127.16.178.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.2.keytab.
I20260806 21:12:08.147182 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.2:0
--local_ip_for_outbound_sockets=127.16.178.2
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:46619
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.2.keytab
--principal=oryx/127.16.178.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:35595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:12:08.264591 29093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:12:08.264873 29093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:12:08.264936 29093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:12:08.269093 29093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:12:08.269181 29093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:12:08.269238 29093 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 21:12:08.269296 29093 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.2
I20260806 21:12:08.274876 29093 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:35595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.2.keytab
--principal=oryx/127.16.178.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.178.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.178.62:46619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29093
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.2
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:12:08.276074 29093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:12:08.277211 29093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:12:08.287372 29103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:08.287333 29107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:08.287577 29093 server_base.cc:1034] running on GCE node
W20260806 21:12:08.287355 29100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:08.288136 29093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:12:08.288807 29093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:12:08.289981 29093 hybrid_clock.cc:648] HybridClock initialized: now 1786050728289956 us; error 28 us; skew 500 ppm
Aug 06 21:12:08 dist-test-slave-lgr5 krb5kdc[28695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050728, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.178.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:12:08.292698 29093 init.cc:377] Logged in from keytab as oryx/127.16.178.2@KRBTEST.COM (short username oryx)
I20260806 21:12:08.293628 29093 webserver.cc:460] Webserver started at http://127.16.178.2:42807/ using document root <none> and password file <none>
I20260806 21:12:08.294174 29093 fs_manager.cc:359] Metadata directory not provided
I20260806 21:12:08.294234 29093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:12:08.294437 29093 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:12:08.296615 29093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-1/data/instance:
uuid: "18cbdda3a4834b2bbae22c4a599e7d3a"
format_stamp: "Formatted at 2026-08-06 21:12:08 on dist-test-slave-lgr5"
server_key: "07e6e339f45ea770ac70d6301f78f925"
server_key_iv: "aaf3ba4b287859b90213a5343fddef24"
server_key_version: "encryptionkey@0"
I20260806 21:12:08.297217 29093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance:
uuid: "18cbdda3a4834b2bbae22c4a599e7d3a"
format_stamp: "Formatted at 2026-08-06 21:12:08 on dist-test-slave-lgr5"
server_key: "07e6e339f45ea770ac70d6301f78f925"
server_key_iv: "aaf3ba4b287859b90213a5343fddef24"
server_key_version: "encryptionkey@0"
I20260806 21:12:08.300990 29093 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 21:12:08.303395 29117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:08.304682 29093 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 21:12:08.304787 29093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-1/wal
uuid: "18cbdda3a4834b2bbae22c4a599e7d3a"
format_stamp: "Formatted at 2026-08-06 21:12:08 on dist-test-slave-lgr5"
server_key: "07e6e339f45ea770ac70d6301f78f925"
server_key_iv: "aaf3ba4b287859b90213a5343fddef24"
server_key_version: "encryptionkey@0"
I20260806 21:12:08.304899 29093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:12:08.322084 29093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:12:08.325191 29093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:12:08.325387 29093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:12:08.327076 29093 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:12:08.327138 29093 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:08.327196 29093 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:12:08.327226 29093 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 21:12:08 dist-test-slave-lgr5 krb5kdc[28695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050728, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.178.2@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM
I20260806 21:12:08.338107 29093 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.2:43593
I20260806 21:12:08.338153 29323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.2:43593 every 8 connection(s)
I20260806 21:12:08.339076 29093 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
I20260806 21:12:08.341887 28837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:08.327818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:33445 (local address 127.16.178.62:46619)
0806 21:12:08.327987 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:08.327993 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:08.328995 (+ 1002us) server_negotiation.cc:407] Connection header received
0806 21:12:08.330213 (+ 1218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:08.330218 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:08.330284 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:08.330345 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:08.332243 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.332997 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.334321 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.334558 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.334653 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:08.338381 (+ 3728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:08.338385 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:08.338388 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:08.338419 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:08.339920 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:08.340505 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:08.340508 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:08.340509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:08.340592 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:08.340871 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:08.340873 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:08.340874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:08.341087 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:08.341178 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:08.341647 (+ 469us) server_negotiation.cc:299] Negotiation successful
0806 21:12:08.341755 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 21:12:08.342756 29137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:08.328258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:46619 (local address 127.16.178.2:33445)
0806 21:12:08.328791 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:08.328851 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:08.329943 (+ 1092us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:08.330825 (+ 882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:08.330835 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:08.331315 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:08.332035 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:08.332058 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.333132 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.333135 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:08.334197 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:08.334210 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.334720 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.334724 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:08.334900 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:08.336274 (+ 1374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:08.336298 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:08.338187 (+ 1889us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:12:08.340057 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:08.340062 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:08.340075 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:08.340369 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:08.340664 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:08.340666 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:08.340667 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:08.340800 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:08.341179 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:12:08.341185 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:12:08.341433 (+ 248us) client_negotiation.cc:769] Sending connection context
0806 21:12:08.341595 (+ 162us) client_negotiation.cc:240] Negotiation successful
0806 21:12:08.341875 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":127,"threads_started":1}
I20260806 21:12:08.344120 29324 heartbeater.cc:347] Connected to a master server at 127.16.178.62:46619
I20260806 21:12:08.344286 29324 heartbeater.cc:464] Registering TS with master...
I20260806 21:12:08.344841 29324 heartbeater.cc:511] Master 127.16.178.62:46619 requested a full tablet report, sending...
I20260806 21:12:08.345413 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 29093
I20260806 21:12:08.345558 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance
I20260806 21:12:08.345896 17096 external_mini_cluster.cc:1428] Setting key 2dccc913de748d5a865afc1a3552d30f
I20260806 21:12:08.345913 28767 ts_manager.cc:194] Registered new tserver with Master: 18cbdda3a4834b2bbae22c4a599e7d3a (127.16.178.2:43593)
I20260806 21:12:08.346649 28767 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.178.2@KRBTEST.COM'} at 127.16.178.2:33445
I20260806 21:12:08.354748 29332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:08.346490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:46619 (local address 127.16.178.2:44003)
0806 21:12:08.346794 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:08.346810 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:08.346940 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:08.347370 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:08.347373 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:08.347612 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:08.347755 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:08.347763 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.348884 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.348888 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:08.349682 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:08.349693 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.350258 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.350260 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:08.350330 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:08.350864 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:08.350879 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:08.351295 (+ 416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:12:08.353142 (+ 1847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:08.353146 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:08.353148 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:08.353356 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:08.353685 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:08.353687 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:08.353689 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:08.353773 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:08.354271 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:12:08.354273 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:12:08.354357 (+ 84us) client_negotiation.cc:769] Sending connection context
0806 21:12:08.354433 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 21:12:08.354578 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":103,"threads_started":1}
I20260806 21:12:08.354815 28837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:08.346834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:44003 (local address 127.16.178.62:46619)
0806 21:12:08.347095 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:08.347100 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:08.347113 (+ 13us) server_negotiation.cc:407] Connection header received
0806 21:12:08.347149 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:08.347152 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:08.347190 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:08.347251 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:08.347907 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.348702 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.349818 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.350143 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.350257 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:08.351428 (+ 1171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:08.351431 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:08.351433 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:08.351458 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:08.352997 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:08.353468 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:08.353472 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:08.353474 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:08.353585 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:08.353855 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:08.353858 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:08.353860 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:08.354183 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:08.354294 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:08.354513 (+ 219us) server_negotiation.cc:299] Negotiation successful
0806 21:12:08.354645 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191}
WARNING: no policy specified for oryx/127.16.178.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.178.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.178.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.3.keytab.
Entry for principal oryx/127.16.178.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.3.keytab.
I20260806 21:12:08.378995 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.3:0
--local_ip_for_outbound_sockets=127.16.178.3
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:46619
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.3.keytab
--principal=oryx/127.16.178.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:35595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:12:08.493760 29336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:12:08.494051 29336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:12:08.494096 29336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:12:08.498123 29336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:12:08.498183 29336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:12:08.498222 29336 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 21:12:08.498262 29336 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.3
I20260806 21:12:08.503383 29336 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:35595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.3.keytab
--principal=oryx/127.16.178.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.178.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.178.62:46619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29336
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.3
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:12:08.504396 29336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:12:08.505420 29336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 21:12:08.515266 29336 server_base.cc:1034] running on GCE node
W20260806 21:12:08.515224 29341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:08.515293 29342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:08.515224 29350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:08.515950 29336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:12:08.516649 29336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:12:08.517817 29336 hybrid_clock.cc:648] HybridClock initialized: now 1786050728517800 us; error 32 us; skew 500 ppm
Aug 06 21:12:08 dist-test-slave-lgr5 krb5kdc[28695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050728, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.178.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:12:08.521032 29336 init.cc:377] Logged in from keytab as oryx/127.16.178.3@KRBTEST.COM (short username oryx)
I20260806 21:12:08.522104 29336 webserver.cc:460] Webserver started at http://127.16.178.3:36563/ using document root <none> and password file <none>
I20260806 21:12:08.522720 29336 fs_manager.cc:359] Metadata directory not provided
I20260806 21:12:08.522796 29336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:12:08.523025 29336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:12:08.525377 29336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-2/data/instance:
uuid: "4b482e8b109143a28f854c2bee2d815f"
format_stamp: "Formatted at 2026-08-06 21:12:08 on dist-test-slave-lgr5"
server_key: "421b7c39720922dfefa4624f87f4599c"
server_key_iv: "b95bcb759a69698c0e9b265c89a776eb"
server_key_version: "encryptionkey@0"
I20260806 21:12:08.525985 29336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance:
uuid: "4b482e8b109143a28f854c2bee2d815f"
format_stamp: "Formatted at 2026-08-06 21:12:08 on dist-test-slave-lgr5"
server_key: "421b7c39720922dfefa4624f87f4599c"
server_key_iv: "b95bcb759a69698c0e9b265c89a776eb"
server_key_version: "encryptionkey@0"
I20260806 21:12:08.530259 29336 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:12:08.533094 29358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:08.534518 29336 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 21:12:08.534670 29336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-2/wal
uuid: "4b482e8b109143a28f854c2bee2d815f"
format_stamp: "Formatted at 2026-08-06 21:12:08 on dist-test-slave-lgr5"
server_key: "421b7c39720922dfefa4624f87f4599c"
server_key_iv: "b95bcb759a69698c0e9b265c89a776eb"
server_key_version: "encryptionkey@0"
I20260806 21:12:08.534806 29336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:12:08.559444 29336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:12:08.562723 29336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:12:08.562933 29336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:12:08.564680 29336 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:12:08.564749 29336 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:08.564808 29336 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:12:08.564838 29336 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 21:12:08 dist-test-slave-lgr5 krb5kdc[28695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050728, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.178.3@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM
I20260806 21:12:08.575726 29336 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.3:44397
I20260806 21:12:08.576573 29530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.3:44397 every 8 connection(s)
I20260806 21:12:08.576783 29336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
I20260806 21:12:08.578574 28837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:08.565597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:33273 (local address 127.16.178.62:46619)
0806 21:12:08.565733 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:08.565737 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:08.566883 (+ 1146us) server_negotiation.cc:407] Connection header received
0806 21:12:08.567989 (+ 1106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:08.567993 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:08.568063 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:08.568160 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:08.569607 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.570185 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.571540 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.571816 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.571916 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:08.574661 (+ 2745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:08.574664 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:08.574666 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:08.574704 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:08.576247 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:08.576947 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:08.576951 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:08.576953 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:08.577037 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:08.577455 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:08.577472 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:08.577474 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:08.577728 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:08.577830 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:08.578298 (+ 468us) server_negotiation.cc:299] Negotiation successful
0806 21:12:08.578413 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 21:12:08.579411 29372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:08.566157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:46619 (local address 127.16.178.3:33273)
0806 21:12:08.566723 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:08.566772 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:08.567717 (+ 945us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:08.568520 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:08.568527 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:08.568891 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:08.569454 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:08.569470 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.570316 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.570319 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:08.571415 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:08.571426 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.571986 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.571990 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:08.572213 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:08.572846 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:08.572863 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:08.574472 (+ 1609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:12:08.576372 (+ 1900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:08.576378 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:08.576396 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:08.576848 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:08.577138 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:08.577141 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:08.577143 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:08.577332 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:08.577844 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:12:08.577849 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:12:08.578097 (+ 248us) client_negotiation.cc:769] Sending connection context
0806 21:12:08.578254 (+ 157us) client_negotiation.cc:240] Negotiation successful
0806 21:12:08.578525 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":150,"threads_started":1}
I20260806 21:12:08.580802 29531 heartbeater.cc:347] Connected to a master server at 127.16.178.62:46619
I20260806 21:12:08.580961 29531 heartbeater.cc:464] Registering TS with master...
I20260806 21:12:08.581566 29531 heartbeater.cc:511] Master 127.16.178.62:46619 requested a full tablet report, sending...
I20260806 21:12:08.582767 28767 ts_manager.cc:194] Registered new tserver with Master: 4b482e8b109143a28f854c2bee2d815f (127.16.178.3:44397)
I20260806 21:12:08.583470 28767 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.178.3@KRBTEST.COM'} at 127.16.178.3:33273
I20260806 21:12:08.587064 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 29336
I20260806 21:12:08.587229 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance
I20260806 21:12:08.587543 17096 external_mini_cluster.cc:1428] Setting key 68315613582308f5c58e4865adde73b6
I20260806 21:12:08.590092 17096 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 21:12:08.590715 28837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:08.583183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:42427 (local address 127.16.178.62:46619)
0806 21:12:08.583309 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:08.583313 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:08.583497 (+ 184us) server_negotiation.cc:407] Connection header received
0806 21:12:08.583622 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:08.583625 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:08.583668 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:08.583740 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:08.584290 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.585051 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.585835 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.586056 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.586138 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:08.587286 (+ 1148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:08.587290 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:08.587294 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:08.587323 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:08.588940 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:08.589397 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:08.589400 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:08.589401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:08.589541 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:08.589824 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:08.589827 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:08.589829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:08.590077 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:08.590169 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:08.590424 (+ 255us) server_negotiation.cc:299] Negotiation successful
0806 21:12:08.590534 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 21:12:08.590729 29539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:08.583085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:46619 (local address 127.16.178.3:42427)
0806 21:12:08.583420 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:08.583433 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:08.583521 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:08.583785 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:08.583789 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:08.584021 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:08.584179 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:08.584186 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.585170 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.585173 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:08.585728 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:08.585736 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.586152 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.586154 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:08.586222 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:08.586679 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:08.586690 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:08.587102 (+ 412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:12:08.589049 (+ 1947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:08.589053 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:08.589055 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:08.589302 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:08.589638 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:08.589642 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:08.589644 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:08.589732 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:08.590174 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:12:08.590177 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:12:08.590269 (+ 92us) client_negotiation.cc:769] Sending connection context
0806 21:12:08.590358 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 21:12:08.590527 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":112,"threads_started":1}
Aug 06 21:12:08 dist-test-slave-lgr5 krb5kdc[28695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050728, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 21:12:08 dist-test-slave-lgr5 krb5kdc[28695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:08 dist-test-slave-lgr5 krb5kdc[28695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
W20260806 21:12:08.616001 28837 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:08.609889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39256 (local address 127.16.178.62:46619)
0806 21:12:08.610047 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:08.610051 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:08.610064 (+ 13us) server_negotiation.cc:407] Connection header received
0806 21:12:08.610107 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:08.610110 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:08.610151 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:08.610221 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:08.611058 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.611594 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.612495 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.612710 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.612799 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:08.615832 (+ 3033us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39256: BlockingRecv error: recv got EOF from 127.0.0.1:39256 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 06 21:12:08 dist-test-slave-lgr5 krb5kdc[28695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050728, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM
I20260806 21:12:08.624977 28837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:08.617465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39258 (local address 127.16.178.62:46619)
0806 21:12:08.617570 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:08.617573 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:08.617707 (+ 134us) server_negotiation.cc:407] Connection header received
0806 21:12:08.617801 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:08.617803 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:08.617843 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:08.617913 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:08.618451 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.618950 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.619728 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.619936 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.620015 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:08.622153 (+ 2138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:08.622156 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:08.622158 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:08.622184 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:08.623370 (+ 1186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:08.623766 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:08.623770 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:08.623772 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:08.623865 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:08.624123 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:08.624126 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:08.624128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:08.624318 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:08.624438 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:08.624662 (+ 224us) server_negotiation.cc:299] Negotiation successful
0806 21:12:08.624794 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 21:12:08.627863 28767 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39258:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:12:08.630986 28767 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:12:08.646272 29567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:08.640593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.2:43593 (local address 127.0.0.1:54552)
0806 21:12:08.641638 (+ 1045us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:08.641651 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:08.641747 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:08.642342 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:08.642345 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:08.642364 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:12:08.642459 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:08.642464 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.643775 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.643779 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:08.644865 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:08.644878 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.645881 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.645885 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:08.645984 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:08.646005 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 21:12:08.646060 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 21:12:08.646117 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":986,"thread_start_us":105,"threads_started":1}
I20260806 21:12:08.646341 29576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:08.641159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54552 (local address 127.16.178.2:43593)
0806 21:12:08.641601 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:08.641613 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:08.641732 (+ 119us) server_negotiation.cc:407] Connection header received
0806 21:12:08.641875 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:08.641879 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:08.642039 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:08.642189 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:12:08.642589 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.643648 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.644987 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.645767 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.645892 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:08.646055 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:08.646112 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 21:12:08.646205 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":120,"threads_started":1}
I20260806 21:12:08.646340 29568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:08.640136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.3:44397 (local address 127.0.0.1:38944)
0806 21:12:08.640957 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:08.641016 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:08.641150 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:08.641875 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:08.641882 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:08.641921 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:12:08.642101 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:08.642109 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.643182 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.643185 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:08.644777 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:08.644790 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.645926 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.645930 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:08.646023 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:08.646048 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 21:12:08.646101 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 21:12:08.646178 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":755,"mutex_wait_us":97,"spinlock_wait_cycles":6656,"thread_start_us":102,"threads_started":1}
I20260806 21:12:08.646814 29572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:08.640140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47160 (local address 127.16.178.1:32881)
0806 21:12:08.640832 (+ 692us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:08.640839 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:08.641105 (+ 266us) server_negotiation.cc:407] Connection header received
0806 21:12:08.641327 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:08.641332 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:08.641554 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:08.641672 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:12:08.642215 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.643212 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.644925 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.645514 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.645621 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:08.645789 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:08.646345 (+ 556us) server_negotiation.cc:299] Negotiation successful
0806 21:12:08.646671 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":99,"threads_started":1}
I20260806 21:12:08.646272 29573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:08.639976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:32881 (local address 127.0.0.1:47160)
0806 21:12:08.640957 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:08.641016 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:08.641162 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:08.641875 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:08.641882 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:08.641919 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:12:08.642091 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:08.642099 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.643328 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.643331 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:08.644780 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:08.644797 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.645640 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.645644 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:08.645761 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:08.645803 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 21:12:08.645926 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 21:12:08.646081 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":805,"mutex_wait_us":22,"spinlock_wait_cycles":6272,"thread_start_us":129,"threads_started":1}
I20260806 21:12:08.647330 29574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:08.640718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38944 (local address 127.16.178.3:44397)
0806 21:12:08.641446 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:08.641457 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:08.641490 (+ 33us) server_negotiation.cc:407] Connection header received
0806 21:12:08.641560 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:08.641564 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:08.641722 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:08.641988 (+ 266us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:12:08.642229 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.643072 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.644928 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.645766 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.646466 (+ 700us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:08.647029 (+ 563us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:08.647081 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 21:12:08.647177 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":119,"threads_started":1}
I20260806 21:12:08.649747 28970 tablet_service.cc:1467] Processing CreateTablet for tablet b7f88db69d114f71afb4f37febd2ad75 (DEFAULT_TABLE table=test-table [id=be677f808c6042d9b788f056738c3a05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:12:08.649889 29207 tablet_service.cc:1467] Processing CreateTablet for tablet b7f88db69d114f71afb4f37febd2ad75 (DEFAULT_TABLE table=test-table [id=be677f808c6042d9b788f056738c3a05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:12:08.649849 29437 tablet_service.cc:1467] Processing CreateTablet for tablet b7f88db69d114f71afb4f37febd2ad75 (DEFAULT_TABLE table=test-table [id=be677f808c6042d9b788f056738c3a05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:12:08.650902 28970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7f88db69d114f71afb4f37febd2ad75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:12:08.650964 29207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7f88db69d114f71afb4f37febd2ad75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:12:08.650964 29437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7f88db69d114f71afb4f37febd2ad75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:12:08.657799 29579 tablet_bootstrap.cc:492] T b7f88db69d114f71afb4f37febd2ad75 P b45a1e0cc11d4f9c9b52563a245e291b: Bootstrap starting.
I20260806 21:12:08.659592 29581 tablet_bootstrap.cc:492] T b7f88db69d114f71afb4f37febd2ad75 P 18cbdda3a4834b2bbae22c4a599e7d3a: Bootstrap starting.
I20260806 21:12:08.660326 29579 tablet_bootstrap.cc:654] T b7f88db69d114f71afb4f37febd2ad75 P b45a1e0cc11d4f9c9b52563a245e291b: Neither blocks nor log segments found. Creating new log.
I20260806 21:12:08.660472 29580 tablet_bootstrap.cc:492] T b7f88db69d114f71afb4f37febd2ad75 P 4b482e8b109143a28f854c2bee2d815f: Bootstrap starting.
I20260806 21:12:08.661154 29579 log.cc:824] T b7f88db69d114f71afb4f37febd2ad75 P b45a1e0cc11d4f9c9b52563a245e291b: Log is configured to *not* fsync() on all Append() calls
I20260806 21:12:08.661950 29581 tablet_bootstrap.cc:654] T b7f88db69d114f71afb4f37febd2ad75 P 18cbdda3a4834b2bbae22c4a599e7d3a: Neither blocks nor log segments found. Creating new log.
I20260806 21:12:08.662647 29580 tablet_bootstrap.cc:654] T b7f88db69d114f71afb4f37febd2ad75 P 4b482e8b109143a28f854c2bee2d815f: Neither blocks nor log segments found. Creating new log.
I20260806 21:12:08.662844 29581 log.cc:824] T b7f88db69d114f71afb4f37febd2ad75 P 18cbdda3a4834b2bbae22c4a599e7d3a: Log is configured to *not* fsync() on all Append() calls
I20260806 21:12:08.663511 29580 log.cc:824] T b7f88db69d114f71afb4f37febd2ad75 P 4b482e8b109143a28f854c2bee2d815f: Log is configured to *not* fsync() on all Append() calls
I20260806 21:12:08.663631 29579 tablet_bootstrap.cc:492] T b7f88db69d114f71afb4f37febd2ad75 P b45a1e0cc11d4f9c9b52563a245e291b: No bootstrap required, opened a new log
I20260806 21:12:08.663834 29579 ts_tablet_manager.cc:1397] T b7f88db69d114f71afb4f37febd2ad75 P b45a1e0cc11d4f9c9b52563a245e291b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 21:12:08.664901 29581 tablet_bootstrap.cc:492] T b7f88db69d114f71afb4f37febd2ad75 P 18cbdda3a4834b2bbae22c4a599e7d3a: No bootstrap required, opened a new log
I20260806 21:12:08.665093 29581 ts_tablet_manager.cc:1397] T b7f88db69d114f71afb4f37febd2ad75 P 18cbdda3a4834b2bbae22c4a599e7d3a: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 21:12:08.665589 29580 tablet_bootstrap.cc:492] T b7f88db69d114f71afb4f37febd2ad75 P 4b482e8b109143a28f854c2bee2d815f: No bootstrap required, opened a new log
I20260806 21:12:08.665784 29580 ts_tablet_manager.cc:1397] T b7f88db69d114f71afb4f37febd2ad75 P 4b482e8b109143a28f854c2bee2d815f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 21:12:08.668025 29581 raft_consensus.cc:348] T b7f88db69d114f71afb4f37febd2ad75 P 18cbdda3a4834b2bbae22c4a599e7d3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } }
I20260806 21:12:08.668315 29581 raft_consensus.cc:374] T b7f88db69d114f71afb4f37febd2ad75 P 18cbdda3a4834b2bbae22c4a599e7d3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:12:08.668373 29581 raft_consensus.cc:729] T b7f88db69d114f71afb4f37febd2ad75 P 18cbdda3a4834b2bbae22c4a599e7d3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18cbdda3a4834b2bbae22c4a599e7d3a, State: Initialized, Role: FOLLOWER
I20260806 21:12:08.668478 29579 raft_consensus.cc:348] T b7f88db69d114f71afb4f37febd2ad75 P b45a1e0cc11d4f9c9b52563a245e291b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } }
I20260806 21:12:08.668741 29579 raft_consensus.cc:374] T b7f88db69d114f71afb4f37febd2ad75 P b45a1e0cc11d4f9c9b52563a245e291b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:12:08.668808 29579 raft_consensus.cc:729] T b7f88db69d114f71afb4f37febd2ad75 P b45a1e0cc11d4f9c9b52563a245e291b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b45a1e0cc11d4f9c9b52563a245e291b, State: Initialized, Role: FOLLOWER
I20260806 21:12:08.668828 29581 consensus_queue.cc:260] T b7f88db69d114f71afb4f37febd2ad75 P 18cbdda3a4834b2bbae22c4a599e7d3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } }
I20260806 21:12:08.669261 29579 consensus_queue.cc:260] T b7f88db69d114f71afb4f37febd2ad75 P b45a1e0cc11d4f9c9b52563a245e291b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } }
I20260806 21:12:08.669590 29324 heartbeater.cc:503] Master 127.16.178.62:46619 was elected leader, sending a full tablet report...
I20260806 21:12:08.669901 29581 ts_tablet_manager.cc:1428] T b7f88db69d114f71afb4f37febd2ad75 P 18cbdda3a4834b2bbae22c4a599e7d3a: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 21:12:08.670161 29579 ts_tablet_manager.cc:1428] T b7f88db69d114f71afb4f37febd2ad75 P b45a1e0cc11d4f9c9b52563a245e291b: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 21:12:08.670490 29073 heartbeater.cc:503] Master 127.16.178.62:46619 was elected leader, sending a full tablet report...
I20260806 21:12:08.670710 29580 raft_consensus.cc:348] T b7f88db69d114f71afb4f37febd2ad75 P 4b482e8b109143a28f854c2bee2d815f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } }
I20260806 21:12:08.670969 29580 raft_consensus.cc:374] T b7f88db69d114f71afb4f37febd2ad75 P 4b482e8b109143a28f854c2bee2d815f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:12:08.671027 29580 raft_consensus.cc:729] T b7f88db69d114f71afb4f37febd2ad75 P 4b482e8b109143a28f854c2bee2d815f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b482e8b109143a28f854c2bee2d815f, State: Initialized, Role: FOLLOWER
I20260806 21:12:08.671424 29580 consensus_queue.cc:260] T b7f88db69d114f71afb4f37febd2ad75 P 4b482e8b109143a28f854c2bee2d815f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } }
I20260806 21:12:08.672231 29531 heartbeater.cc:503] Master 127.16.178.62:46619 was elected leader, sending a full tablet report...
I20260806 21:12:08.672413 29580 ts_tablet_manager.cc:1428] T b7f88db69d114f71afb4f37febd2ad75 P 4b482e8b109143a28f854c2bee2d815f: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260806 21:12:08.702783 29586 raft_consensus.cc:482] T b7f88db69d114f71afb4f37febd2ad75 P b45a1e0cc11d4f9c9b52563a245e291b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:12:08.703047 29586 raft_consensus.cc:504] T b7f88db69d114f71afb4f37febd2ad75 P b45a1e0cc11d4f9c9b52563a245e291b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } }
I20260806 21:12:08.704320 29586 leader_election.cc:288] T b7f88db69d114f71afb4f37febd2ad75 P b45a1e0cc11d4f9c9b52563a245e291b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b482e8b109143a28f854c2bee2d815f (127.16.178.3:44397), 18cbdda3a4834b2bbae22c4a599e7d3a (127.16.178.2:43593)
I20260806 21:12:08.709985 29576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:08.704760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:36183 (local address 127.16.178.2:43593)
0806 21:12:08.704917 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:08.704922 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:08.705069 (+ 147us) server_negotiation.cc:407] Connection header received
0806 21:12:08.705252 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:08.705255 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:08.705307 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:08.705384 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:12:08.705825 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.706658 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.708607 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.709300 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.709439 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:08.709507 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:08.709744 (+ 237us) server_negotiation.cc:299] Negotiation successful
0806 21:12:08.709821 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 21:12:08.709978 29574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:08.704660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:44923 (local address 127.16.178.3:44397)
0806 21:12:08.704804 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:08.704810 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:08.705082 (+ 272us) server_negotiation.cc:407] Connection header received
0806 21:12:08.705307 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:08.705311 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:08.705359 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:08.705432 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:12:08.705824 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.706668 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.708607 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.709300 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.709501 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:08.709577 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:08.709741 (+ 164us) server_negotiation.cc:299] Negotiation successful
0806 21:12:08.709821 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 21:12:08.710034 29593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:08.704660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.2:43593 (local address 127.16.178.1:36183)
0806 21:12:08.704983 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:08.704995 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:08.705145 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:08.705414 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:08.705417 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:08.705535 (+ 118us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:12:08.705708 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:08.705716 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.706833 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.706837 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:08.708436 (+ 1599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:08.708456 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.709530 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.709533 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:08.709627 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:08.709644 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 21:12:08.709724 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 21:12:08.709833 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"mutex_wait_us":93,"spinlock_wait_cycles":42368,"thread_start_us":121,"threads_started":1}
I20260806 21:12:08.710034 29588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:08.704540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.3:44397 (local address 127.16.178.1:44923)
0806 21:12:08.704978 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:08.704993 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:08.705146 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:08.705455 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:08.705484 (+ 29us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:08.705505 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:12:08.705708 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:08.705716 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.706833 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.706838 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:08.708452 (+ 1614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:08.708459 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.709443 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.709447 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:08.709607 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:08.709640 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 21:12:08.709724 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 21:12:08.709833 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"mutex_wait_us":201,"spinlock_wait_cycles":17152,"thread_start_us":155,"threads_started":1}
I20260806 21:12:08.710922 29464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7f88db69d114f71afb4f37febd2ad75" candidate_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b482e8b109143a28f854c2bee2d815f" is_pre_election: true
I20260806 21:12:08.710917 29246 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7f88db69d114f71afb4f37febd2ad75" candidate_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" is_pre_election: true
I20260806 21:12:08.711230 29246 raft_consensus.cc:2454] T b7f88db69d114f71afb4f37febd2ad75 P 18cbdda3a4834b2bbae22c4a599e7d3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b45a1e0cc11d4f9c9b52563a245e291b in term 0.
I20260806 21:12:08.711234 29464 raft_consensus.cc:2454] T b7f88db69d114f71afb4f37febd2ad75 P 4b482e8b109143a28f854c2bee2d815f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b45a1e0cc11d4f9c9b52563a245e291b in term 0.
I20260806 21:12:08.711745 28896 leader_election.cc:302] T b7f88db69d114f71afb4f37febd2ad75 P b45a1e0cc11d4f9c9b52563a245e291b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b482e8b109143a28f854c2bee2d815f, b45a1e0cc11d4f9c9b52563a245e291b; no voters:
I20260806 21:12:08.712036 29586 raft_consensus.cc:2781] T b7f88db69d114f71afb4f37febd2ad75 P b45a1e0cc11d4f9c9b52563a245e291b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:12:08.712145 29586 raft_consensus.cc:482] T b7f88db69d114f71afb4f37febd2ad75 P b45a1e0cc11d4f9c9b52563a245e291b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:12:08.712193 29586 raft_consensus.cc:3037] T b7f88db69d114f71afb4f37febd2ad75 P b45a1e0cc11d4f9c9b52563a245e291b [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:12:08.713264 29586 raft_consensus.cc:504] T b7f88db69d114f71afb4f37febd2ad75 P b45a1e0cc11d4f9c9b52563a245e291b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } }
I20260806 21:12:08.713798 29586 leader_election.cc:288] T b7f88db69d114f71afb4f37febd2ad75 P b45a1e0cc11d4f9c9b52563a245e291b [CANDIDATE]: Term 1 election: Requested vote from peers 4b482e8b109143a28f854c2bee2d815f (127.16.178.3:44397), 18cbdda3a4834b2bbae22c4a599e7d3a (127.16.178.2:43593)
I20260806 21:12:08.714195 29246 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7f88db69d114f71afb4f37febd2ad75" candidate_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a"
I20260806 21:12:08.714195 29464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7f88db69d114f71afb4f37febd2ad75" candidate_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b482e8b109143a28f854c2bee2d815f"
I20260806 21:12:08.714316 29246 raft_consensus.cc:3037] T b7f88db69d114f71afb4f37febd2ad75 P 18cbdda3a4834b2bbae22c4a599e7d3a [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:12:08.714313 29464 raft_consensus.cc:3037] T b7f88db69d114f71afb4f37febd2ad75 P 4b482e8b109143a28f854c2bee2d815f [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:12:08.715369 29464 raft_consensus.cc:2454] T b7f88db69d114f71afb4f37febd2ad75 P 4b482e8b109143a28f854c2bee2d815f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b45a1e0cc11d4f9c9b52563a245e291b in term 1.
I20260806 21:12:08.715687 29246 raft_consensus.cc:2454] T b7f88db69d114f71afb4f37febd2ad75 P 18cbdda3a4834b2bbae22c4a599e7d3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b45a1e0cc11d4f9c9b52563a245e291b in term 1.
I20260806 21:12:08.715684 28896 leader_election.cc:302] T b7f88db69d114f71afb4f37febd2ad75 P b45a1e0cc11d4f9c9b52563a245e291b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b482e8b109143a28f854c2bee2d815f, b45a1e0cc11d4f9c9b52563a245e291b; no voters:
I20260806 21:12:08.715847 29586 raft_consensus.cc:2781] T b7f88db69d114f71afb4f37febd2ad75 P b45a1e0cc11d4f9c9b52563a245e291b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:12:08.716068 29586 raft_consensus.cc:686] T b7f88db69d114f71afb4f37febd2ad75 P b45a1e0cc11d4f9c9b52563a245e291b [term 1 LEADER]: Becoming Leader. State: Replica: b45a1e0cc11d4f9c9b52563a245e291b, State: Running, Role: LEADER
I20260806 21:12:08.716436 29586 consensus_queue.cc:237] T b7f88db69d114f71afb4f37febd2ad75 P b45a1e0cc11d4f9c9b52563a245e291b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } }
I20260806 21:12:08.719699 28767 catalog_manager.cc:5495] T b7f88db69d114f71afb4f37febd2ad75 P b45a1e0cc11d4f9c9b52563a245e291b reported cstate change: term changed from 0 to 1, leader changed from <none> to b45a1e0cc11d4f9c9b52563a245e291b (127.16.178.1). New cstate: current_term: 1 leader_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } health_report { overall_health: UNKNOWN } } }
I20260806 21:12:08.747007 29572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:08.743484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47176 (local address 127.16.178.1:32881)
0806 21:12:08.743640 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:08.743644 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:08.743658 (+ 14us) server_negotiation.cc:407] Connection header received
0806 21:12:08.743750 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:08.743752 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:08.743809 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:08.743911 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:12:08.744179 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.744691 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.745558 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.745769 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.745856 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:08.746131 (+ 275us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:12:08.746602 (+ 471us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:12:08.746709 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:08.746800 (+ 91us) server_negotiation.cc:299] Negotiation successful
0806 21:12:08.746862 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 21:12:08.753252 29246 raft_consensus.cc:1261] T b7f88db69d114f71afb4f37febd2ad75 P 18cbdda3a4834b2bbae22c4a599e7d3a [term 1 FOLLOWER]: Refusing update from remote peer b45a1e0cc11d4f9c9b52563a245e291b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:12:08.753716 29464 raft_consensus.cc:1261] T b7f88db69d114f71afb4f37febd2ad75 P 4b482e8b109143a28f854c2bee2d815f [term 1 FOLLOWER]: Refusing update from remote peer b45a1e0cc11d4f9c9b52563a245e291b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:12:08.754091 29586 consensus_queue.cc:1035] T b7f88db69d114f71afb4f37febd2ad75 P b45a1e0cc11d4f9c9b52563a245e291b [LEADER]: Connected to new peer: Peer: permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:12:08.754329 29597 consensus_queue.cc:1035] T b7f88db69d114f71afb4f37febd2ad75 P b45a1e0cc11d4f9c9b52563a245e291b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:12:08.762629 29602 mvcc.cc:205] Tried to move back new op lower bound from 7315663784964898816 to 7315663784825253888. Current Snapshot: MvccSnapshot[applied={T|T < 7315663784964898816}]
I20260806 21:12:08.772377 28767 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39258:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:12:08.772635 28767 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39258:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:12:08.775663 28767 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9f793d6ef89b4cd0a78f73c7c77df3f9: Sending DeleteTablet for 3 replicas of tablet b7f88db69d114f71afb4f37febd2ad75
I20260806 21:12:08.776670 28970 tablet_service.cc:1514] Processing DeleteTablet for tablet b7f88db69d114f71afb4f37febd2ad75 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:12:08 UTC) from {username='oryx', principal='oryx/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:47160
I20260806 21:12:08.776697 29437 tablet_service.cc:1514] Processing DeleteTablet for tablet b7f88db69d114f71afb4f37febd2ad75 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:12:08 UTC) from {username='oryx', principal='oryx/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:38944
I20260806 21:12:08.776836 29207 tablet_service.cc:1514] Processing DeleteTablet for tablet b7f88db69d114f71afb4f37febd2ad75 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:12:08 UTC) from {username='oryx', principal='oryx/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:54552
I20260806 21:12:08.777305 29620 tablet_replica.cc:330] T b7f88db69d114f71afb4f37febd2ad75 P 4b482e8b109143a28f854c2bee2d815f: stopping tablet replica
I20260806 21:12:08.777562 29618 tablet_replica.cc:330] T b7f88db69d114f71afb4f37febd2ad75 P b45a1e0cc11d4f9c9b52563a245e291b: stopping tablet replica
I20260806 21:12:08.777792 29620 raft_consensus.cc:2229] T b7f88db69d114f71afb4f37febd2ad75 P 4b482e8b109143a28f854c2bee2d815f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:12:08.777900 29618 raft_consensus.cc:2229] T b7f88db69d114f71afb4f37febd2ad75 P b45a1e0cc11d4f9c9b52563a245e291b [term 1 LEADER]: Raft consensus shutting down.
I20260806 21:12:08.778180 29620 raft_consensus.cc:2258] T b7f88db69d114f71afb4f37febd2ad75 P 4b482e8b109143a28f854c2bee2d815f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:12:08.778339 29618 raft_consensus.cc:2258] T b7f88db69d114f71afb4f37febd2ad75 P b45a1e0cc11d4f9c9b52563a245e291b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:12:08.778317 28767 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39258:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 21:12:08.778764 29622 tablet_replica.cc:330] T b7f88db69d114f71afb4f37febd2ad75 P 18cbdda3a4834b2bbae22c4a599e7d3a: stopping tablet replica
W20260806 21:12:08.778844 28767 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:12:08.779556 29622 raft_consensus.cc:2229] T b7f88db69d114f71afb4f37febd2ad75 P 18cbdda3a4834b2bbae22c4a599e7d3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:12:08.779938 29622 raft_consensus.cc:2258] T b7f88db69d114f71afb4f37febd2ad75 P 18cbdda3a4834b2bbae22c4a599e7d3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:12:08.780983 29620 ts_tablet_manager.cc:1905] T b7f88db69d114f71afb4f37febd2ad75 P 4b482e8b109143a28f854c2bee2d815f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:12:08.782323 29622 ts_tablet_manager.cc:1905] T b7f88db69d114f71afb4f37febd2ad75 P 18cbdda3a4834b2bbae22c4a599e7d3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:12:08.784704 29620 ts_tablet_manager.cc:1918] T b7f88db69d114f71afb4f37febd2ad75 P 4b482e8b109143a28f854c2bee2d815f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:12:08.785437 29622 ts_tablet_manager.cc:1918] T b7f88db69d114f71afb4f37febd2ad75 P 18cbdda3a4834b2bbae22c4a599e7d3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:12:08.785594 29622 log.cc:1196] T b7f88db69d114f71afb4f37febd2ad75 P 18cbdda3a4834b2bbae22c4a599e7d3a: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-1/wal/wals/b7f88db69d114f71afb4f37febd2ad75
I20260806 21:12:08.785758 29620 log.cc:1196] T b7f88db69d114f71afb4f37febd2ad75 P 4b482e8b109143a28f854c2bee2d815f: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-2/wal/wals/b7f88db69d114f71afb4f37febd2ad75
I20260806 21:12:08.785933 29622 ts_tablet_manager.cc:1939] T b7f88db69d114f71afb4f37febd2ad75 P 18cbdda3a4834b2bbae22c4a599e7d3a: Deleting consensus metadata
I20260806 21:12:08.786381 29620 ts_tablet_manager.cc:1939] T b7f88db69d114f71afb4f37febd2ad75 P 4b482e8b109143a28f854c2bee2d815f: Deleting consensus metadata
I20260806 21:12:08.787124 29618 ts_tablet_manager.cc:1905] T b7f88db69d114f71afb4f37febd2ad75 P b45a1e0cc11d4f9c9b52563a245e291b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:12:08.787599 28742 catalog_manager.cc:4841] TS 18cbdda3a4834b2bbae22c4a599e7d3a (127.16.178.2:43593): tablet b7f88db69d114f71afb4f37febd2ad75 (table test-table [id=be677f808c6042d9b788f056738c3a05]) successfully deleted
I20260806 21:12:08.787732 29437 tablet_service.cc:1467] Processing CreateTablet for tablet 07e2f1592efc43548b02aa5b31003a92 (DEFAULT_TABLE table=test-table [id=dad7b49d3e274e1d92fee95562a36a5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:12:08.788008 29437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07e2f1592efc43548b02aa5b31003a92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:12:08.788198 29207 tablet_service.cc:1467] Processing CreateTablet for tablet 07e2f1592efc43548b02aa5b31003a92 (DEFAULT_TABLE table=test-table [id=dad7b49d3e274e1d92fee95562a36a5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:12:08.788476 29207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07e2f1592efc43548b02aa5b31003a92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:12:08.787082 28970 tablet_service.cc:1467] Processing CreateTablet for tablet 07e2f1592efc43548b02aa5b31003a92 (DEFAULT_TABLE table=test-table [id=dad7b49d3e274e1d92fee95562a36a5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:12:08.790480 28970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07e2f1592efc43548b02aa5b31003a92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:12:08.790884 28741 catalog_manager.cc:4841] TS 4b482e8b109143a28f854c2bee2d815f (127.16.178.3:44397): tablet b7f88db69d114f71afb4f37febd2ad75 (table test-table [id=be677f808c6042d9b788f056738c3a05]) successfully deleted
I20260806 21:12:08.791826 29581 tablet_bootstrap.cc:492] T 07e2f1592efc43548b02aa5b31003a92 P 18cbdda3a4834b2bbae22c4a599e7d3a: Bootstrap starting.
I20260806 21:12:08.792142 29618 ts_tablet_manager.cc:1918] T b7f88db69d114f71afb4f37febd2ad75 P b45a1e0cc11d4f9c9b52563a245e291b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:12:08.792392 29618 log.cc:1196] T b7f88db69d114f71afb4f37febd2ad75 P b45a1e0cc11d4f9c9b52563a245e291b: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-0/wal/wals/b7f88db69d114f71afb4f37febd2ad75
I20260806 21:12:08.792779 29618 ts_tablet_manager.cc:1939] T b7f88db69d114f71afb4f37febd2ad75 P b45a1e0cc11d4f9c9b52563a245e291b: Deleting consensus metadata
I20260806 21:12:08.792876 29581 tablet_bootstrap.cc:654] T 07e2f1592efc43548b02aa5b31003a92 P 18cbdda3a4834b2bbae22c4a599e7d3a: Neither blocks nor log segments found. Creating new log.
I20260806 21:12:08.793556 29579 tablet_bootstrap.cc:492] T 07e2f1592efc43548b02aa5b31003a92 P b45a1e0cc11d4f9c9b52563a245e291b: Bootstrap starting.
I20260806 21:12:08.794152 28743 catalog_manager.cc:4841] TS b45a1e0cc11d4f9c9b52563a245e291b (127.16.178.1:32881): tablet b7f88db69d114f71afb4f37febd2ad75 (table test-table [id=be677f808c6042d9b788f056738c3a05]) successfully deleted
I20260806 21:12:08.794430 29580 tablet_bootstrap.cc:492] T 07e2f1592efc43548b02aa5b31003a92 P 4b482e8b109143a28f854c2bee2d815f: Bootstrap starting.
I20260806 21:12:08.794519 29579 tablet_bootstrap.cc:654] T 07e2f1592efc43548b02aa5b31003a92 P b45a1e0cc11d4f9c9b52563a245e291b: Neither blocks nor log segments found. Creating new log.
I20260806 21:12:08.795467 29581 tablet_bootstrap.cc:492] T 07e2f1592efc43548b02aa5b31003a92 P 18cbdda3a4834b2bbae22c4a599e7d3a: No bootstrap required, opened a new log
I20260806 21:12:08.795519 29580 tablet_bootstrap.cc:654] T 07e2f1592efc43548b02aa5b31003a92 P 4b482e8b109143a28f854c2bee2d815f: Neither blocks nor log segments found. Creating new log.
I20260806 21:12:08.795555 29581 ts_tablet_manager.cc:1397] T 07e2f1592efc43548b02aa5b31003a92 P 18cbdda3a4834b2bbae22c4a599e7d3a: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 21:12:08.795670 29579 tablet_bootstrap.cc:492] T 07e2f1592efc43548b02aa5b31003a92 P b45a1e0cc11d4f9c9b52563a245e291b: No bootstrap required, opened a new log
I20260806 21:12:08.795743 29579 ts_tablet_manager.cc:1397] T 07e2f1592efc43548b02aa5b31003a92 P b45a1e0cc11d4f9c9b52563a245e291b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 21:12:08.796052 29581 raft_consensus.cc:348] T 07e2f1592efc43548b02aa5b31003a92 P 18cbdda3a4834b2bbae22c4a599e7d3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } }
I20260806 21:12:08.796201 29581 raft_consensus.cc:374] T 07e2f1592efc43548b02aa5b31003a92 P 18cbdda3a4834b2bbae22c4a599e7d3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:12:08.796233 29581 raft_consensus.cc:729] T 07e2f1592efc43548b02aa5b31003a92 P 18cbdda3a4834b2bbae22c4a599e7d3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18cbdda3a4834b2bbae22c4a599e7d3a, State: Initialized, Role: FOLLOWER
I20260806 21:12:08.796221 29579 raft_consensus.cc:348] T 07e2f1592efc43548b02aa5b31003a92 P b45a1e0cc11d4f9c9b52563a245e291b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } }
I20260806 21:12:08.796346 29579 raft_consensus.cc:374] T 07e2f1592efc43548b02aa5b31003a92 P b45a1e0cc11d4f9c9b52563a245e291b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:12:08.796375 29579 raft_consensus.cc:729] T 07e2f1592efc43548b02aa5b31003a92 P b45a1e0cc11d4f9c9b52563a245e291b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b45a1e0cc11d4f9c9b52563a245e291b, State: Initialized, Role: FOLLOWER
I20260806 21:12:08.796375 29581 consensus_queue.cc:260] T 07e2f1592efc43548b02aa5b31003a92 P 18cbdda3a4834b2bbae22c4a599e7d3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } }
I20260806 21:12:08.796494 29579 consensus_queue.cc:260] T 07e2f1592efc43548b02aa5b31003a92 P b45a1e0cc11d4f9c9b52563a245e291b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } }
I20260806 21:12:08.796639 29581 ts_tablet_manager.cc:1428] T 07e2f1592efc43548b02aa5b31003a92 P 18cbdda3a4834b2bbae22c4a599e7d3a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 21:12:08.796746 29579 ts_tablet_manager.cc:1428] T 07e2f1592efc43548b02aa5b31003a92 P b45a1e0cc11d4f9c9b52563a245e291b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 21:12:08.798334 29580 tablet_bootstrap.cc:492] T 07e2f1592efc43548b02aa5b31003a92 P 4b482e8b109143a28f854c2bee2d815f: No bootstrap required, opened a new log
I20260806 21:12:08.798437 29580 ts_tablet_manager.cc:1397] T 07e2f1592efc43548b02aa5b31003a92 P 4b482e8b109143a28f854c2bee2d815f: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 21:12:08.798899 29580 raft_consensus.cc:348] T 07e2f1592efc43548b02aa5b31003a92 P 4b482e8b109143a28f854c2bee2d815f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } }
I20260806 21:12:08.799039 29580 raft_consensus.cc:374] T 07e2f1592efc43548b02aa5b31003a92 P 4b482e8b109143a28f854c2bee2d815f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:12:08.799082 29580 raft_consensus.cc:729] T 07e2f1592efc43548b02aa5b31003a92 P 4b482e8b109143a28f854c2bee2d815f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b482e8b109143a28f854c2bee2d815f, State: Initialized, Role: FOLLOWER
I20260806 21:12:08.799216 29580 consensus_queue.cc:260] T 07e2f1592efc43548b02aa5b31003a92 P 4b482e8b109143a28f854c2bee2d815f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } }
I20260806 21:12:08.799508 29580 ts_tablet_manager.cc:1428] T 07e2f1592efc43548b02aa5b31003a92 P 4b482e8b109143a28f854c2bee2d815f: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
W20260806 21:12:08.827893 29532 tablet.cc:2369] T 07e2f1592efc43548b02aa5b31003a92 P 4b482e8b109143a28f854c2bee2d815f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 21:12:08.840337 29326 tablet.cc:2369] T 07e2f1592efc43548b02aa5b31003a92 P 18cbdda3a4834b2bbae22c4a599e7d3a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 21:12:08.847851 29597 raft_consensus.cc:482] T 07e2f1592efc43548b02aa5b31003a92 P b45a1e0cc11d4f9c9b52563a245e291b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:12:08.848012 29597 raft_consensus.cc:504] T 07e2f1592efc43548b02aa5b31003a92 P b45a1e0cc11d4f9c9b52563a245e291b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } }
I20260806 21:12:08.848557 29597 leader_election.cc:288] T 07e2f1592efc43548b02aa5b31003a92 P b45a1e0cc11d4f9c9b52563a245e291b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18cbdda3a4834b2bbae22c4a599e7d3a (127.16.178.2:43593), 4b482e8b109143a28f854c2bee2d815f (127.16.178.3:44397)
I20260806 21:12:08.849001 29464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07e2f1592efc43548b02aa5b31003a92" candidate_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b482e8b109143a28f854c2bee2d815f" is_pre_election: true
I20260806 21:12:08.848989 29246 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07e2f1592efc43548b02aa5b31003a92" candidate_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" is_pre_election: true
I20260806 21:12:08.849148 29246 raft_consensus.cc:2454] T 07e2f1592efc43548b02aa5b31003a92 P 18cbdda3a4834b2bbae22c4a599e7d3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b45a1e0cc11d4f9c9b52563a245e291b in term 0.
I20260806 21:12:08.849152 29464 raft_consensus.cc:2454] T 07e2f1592efc43548b02aa5b31003a92 P 4b482e8b109143a28f854c2bee2d815f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b45a1e0cc11d4f9c9b52563a245e291b in term 0.
I20260806 21:12:08.849498 28898 leader_election.cc:302] T 07e2f1592efc43548b02aa5b31003a92 P b45a1e0cc11d4f9c9b52563a245e291b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18cbdda3a4834b2bbae22c4a599e7d3a, b45a1e0cc11d4f9c9b52563a245e291b; no voters:
I20260806 21:12:08.849670 29597 raft_consensus.cc:2781] T 07e2f1592efc43548b02aa5b31003a92 P b45a1e0cc11d4f9c9b52563a245e291b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:12:08.849733 29597 raft_consensus.cc:482] T 07e2f1592efc43548b02aa5b31003a92 P b45a1e0cc11d4f9c9b52563a245e291b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:12:08.849779 29597 raft_consensus.cc:3037] T 07e2f1592efc43548b02aa5b31003a92 P b45a1e0cc11d4f9c9b52563a245e291b [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:12:08.850780 29597 raft_consensus.cc:504] T 07e2f1592efc43548b02aa5b31003a92 P b45a1e0cc11d4f9c9b52563a245e291b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } }
I20260806 21:12:08.851126 29597 leader_election.cc:288] T 07e2f1592efc43548b02aa5b31003a92 P b45a1e0cc11d4f9c9b52563a245e291b [CANDIDATE]: Term 1 election: Requested vote from peers 18cbdda3a4834b2bbae22c4a599e7d3a (127.16.178.2:43593), 4b482e8b109143a28f854c2bee2d815f (127.16.178.3:44397)
I20260806 21:12:08.851529 29464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07e2f1592efc43548b02aa5b31003a92" candidate_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b482e8b109143a28f854c2bee2d815f"
I20260806 21:12:08.851613 29246 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07e2f1592efc43548b02aa5b31003a92" candidate_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a"
I20260806 21:12:08.851670 29464 raft_consensus.cc:3037] T 07e2f1592efc43548b02aa5b31003a92 P 4b482e8b109143a28f854c2bee2d815f [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:12:08.851743 29246 raft_consensus.cc:3037] T 07e2f1592efc43548b02aa5b31003a92 P 18cbdda3a4834b2bbae22c4a599e7d3a [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:12:08.852823 29246 raft_consensus.cc:2454] T 07e2f1592efc43548b02aa5b31003a92 P 18cbdda3a4834b2bbae22c4a599e7d3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b45a1e0cc11d4f9c9b52563a245e291b in term 1.
I20260806 21:12:08.852850 29464 raft_consensus.cc:2454] T 07e2f1592efc43548b02aa5b31003a92 P 4b482e8b109143a28f854c2bee2d815f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b45a1e0cc11d4f9c9b52563a245e291b in term 1.
I20260806 21:12:08.853124 28896 leader_election.cc:302] T 07e2f1592efc43548b02aa5b31003a92 P b45a1e0cc11d4f9c9b52563a245e291b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b482e8b109143a28f854c2bee2d815f, b45a1e0cc11d4f9c9b52563a245e291b; no voters:
I20260806 21:12:08.853309 29597 raft_consensus.cc:2781] T 07e2f1592efc43548b02aa5b31003a92 P b45a1e0cc11d4f9c9b52563a245e291b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:12:08.853411 29597 raft_consensus.cc:686] T 07e2f1592efc43548b02aa5b31003a92 P b45a1e0cc11d4f9c9b52563a245e291b [term 1 LEADER]: Becoming Leader. State: Replica: b45a1e0cc11d4f9c9b52563a245e291b, State: Running, Role: LEADER
I20260806 21:12:08.853585 29597 consensus_queue.cc:237] T 07e2f1592efc43548b02aa5b31003a92 P b45a1e0cc11d4f9c9b52563a245e291b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } }
I20260806 21:12:08.855345 28762 catalog_manager.cc:5495] T 07e2f1592efc43548b02aa5b31003a92 P b45a1e0cc11d4f9c9b52563a245e291b reported cstate change: term changed from 0 to 1, leader changed from <none> to b45a1e0cc11d4f9c9b52563a245e291b (127.16.178.1). New cstate: current_term: 1 leader_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } health_report { overall_health: UNKNOWN } } }
W20260806 21:12:08.866112 29075 tablet.cc:2369] T 07e2f1592efc43548b02aa5b31003a92 P b45a1e0cc11d4f9c9b52563a245e291b: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 21:12:08 dist-test-slave-lgr5 krb5kdc[28695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050727, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.178.62@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM
I20260806 21:12:08.903247 29636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:08.894359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:46619 (local address 127.0.0.1:39268)
0806 21:12:08.894668 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:08.894684 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:08.894801 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:08.895069 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:08.895071 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:08.895442 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:08.895537 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:08.895543 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.896602 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.896605 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:08.897149 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:08.897156 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.897740 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.897742 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:08.897799 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:08.898349 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:08.898361 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:08.900028 (+ 1667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:12:08.901787 (+ 1759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:08.901795 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:08.901801 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:08.902033 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:08.902309 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:08.902311 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:08.902312 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:08.902380 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:08.902787 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:12:08.902793 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:12:08.902895 (+ 102us) client_negotiation.cc:769] Sending connection context
0806 21:12:08.902979 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 21:12:08.903091 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":137,"threads_started":1}
I20260806 21:12:08.903405 28837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:08.894438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39268 (local address 127.16.178.62:46619)
0806 21:12:08.894585 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:08.894590 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:08.894770 (+ 180us) server_negotiation.cc:407] Connection header received
0806 21:12:08.894881 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:08.894884 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:08.894936 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:08.895060 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:08.895662 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.896502 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.897264 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:08.897630 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:08.897748 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:08.900191 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:08.900195 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:08.900198 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:08.900224 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:08.901676 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:08.902127 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:08.902131 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:08.902132 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:08.902226 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:08.902456 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:08.902459 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:08.902461 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:08.902689 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:08.902789 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:08.903074 (+ 285us) server_negotiation.cc:299] Negotiation successful
0806 21:12:08.903210 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 21:12:08.907800 28762 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:39268:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 21:12:08.908663 28762 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:12:08.913772 28970 tablet_service.cc:1467] Processing CreateTablet for tablet 5e363998298e41d69f61fb9852c3892d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8210799ee3ff46dca5efa3a645c358ce]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 21:12:08.913892 29437 tablet_service.cc:1467] Processing CreateTablet for tablet 5e363998298e41d69f61fb9852c3892d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8210799ee3ff46dca5efa3a645c358ce]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 21:12:08.914224 29437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e363998298e41d69f61fb9852c3892d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:12:08.914225 28970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e363998298e41d69f61fb9852c3892d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:12:08.916878 29207 tablet_service.cc:1467] Processing CreateTablet for tablet 5e363998298e41d69f61fb9852c3892d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8210799ee3ff46dca5efa3a645c358ce]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 21:12:08.917279 29207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e363998298e41d69f61fb9852c3892d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:12:08.917407 29579 tablet_bootstrap.cc:492] T 5e363998298e41d69f61fb9852c3892d P b45a1e0cc11d4f9c9b52563a245e291b: Bootstrap starting.
I20260806 21:12:08.917407 29580 tablet_bootstrap.cc:492] T 5e363998298e41d69f61fb9852c3892d P 4b482e8b109143a28f854c2bee2d815f: Bootstrap starting.
I20260806 21:12:08.918337 29580 tablet_bootstrap.cc:654] T 5e363998298e41d69f61fb9852c3892d P 4b482e8b109143a28f854c2bee2d815f: Neither blocks nor log segments found. Creating new log.
I20260806 21:12:08.918337 29579 tablet_bootstrap.cc:654] T 5e363998298e41d69f61fb9852c3892d P b45a1e0cc11d4f9c9b52563a245e291b: Neither blocks nor log segments found. Creating new log.
I20260806 21:12:08.920351 29581 tablet_bootstrap.cc:492] T 5e363998298e41d69f61fb9852c3892d P 18cbdda3a4834b2bbae22c4a599e7d3a: Bootstrap starting.
I20260806 21:12:08.920439 29580 tablet_bootstrap.cc:492] T 5e363998298e41d69f61fb9852c3892d P 4b482e8b109143a28f854c2bee2d815f: No bootstrap required, opened a new log
I20260806 21:12:08.920524 29580 ts_tablet_manager.cc:1397] T 5e363998298e41d69f61fb9852c3892d P 4b482e8b109143a28f854c2bee2d815f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 21:12:08.920965 29580 raft_consensus.cc:348] T 5e363998298e41d69f61fb9852c3892d P 4b482e8b109143a28f854c2bee2d815f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } }
I20260806 21:12:08.921099 29580 raft_consensus.cc:374] T 5e363998298e41d69f61fb9852c3892d P 4b482e8b109143a28f854c2bee2d815f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:12:08.921133 29580 raft_consensus.cc:729] T 5e363998298e41d69f61fb9852c3892d P 4b482e8b109143a28f854c2bee2d815f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b482e8b109143a28f854c2bee2d815f, State: Initialized, Role: FOLLOWER
I20260806 21:12:08.921312 29581 tablet_bootstrap.cc:654] T 5e363998298e41d69f61fb9852c3892d P 18cbdda3a4834b2bbae22c4a599e7d3a: Neither blocks nor log segments found. Creating new log.
I20260806 21:12:08.921260 29580 consensus_queue.cc:260] T 5e363998298e41d69f61fb9852c3892d P 4b482e8b109143a28f854c2bee2d815f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } }
I20260806 21:12:08.921314 29579 tablet_bootstrap.cc:492] T 5e363998298e41d69f61fb9852c3892d P b45a1e0cc11d4f9c9b52563a245e291b: No bootstrap required, opened a new log
I20260806 21:12:08.921391 29579 ts_tablet_manager.cc:1397] T 5e363998298e41d69f61fb9852c3892d P b45a1e0cc11d4f9c9b52563a245e291b: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 21:12:08.921541 29580 ts_tablet_manager.cc:1428] T 5e363998298e41d69f61fb9852c3892d P 4b482e8b109143a28f854c2bee2d815f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 21:12:08.921761 29579 raft_consensus.cc:348] T 5e363998298e41d69f61fb9852c3892d P b45a1e0cc11d4f9c9b52563a245e291b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } }
I20260806 21:12:08.921860 29579 raft_consensus.cc:374] T 5e363998298e41d69f61fb9852c3892d P b45a1e0cc11d4f9c9b52563a245e291b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:12:08.921793 29587 tablet_replica.cc:439] T 5e363998298e41d69f61fb9852c3892d P 4b482e8b109143a28f854c2bee2d815f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } } }
I20260806 21:12:08.921895 29579 raft_consensus.cc:729] T 5e363998298e41d69f61fb9852c3892d P b45a1e0cc11d4f9c9b52563a245e291b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b45a1e0cc11d4f9c9b52563a245e291b, State: Initialized, Role: FOLLOWER
I20260806 21:12:08.921949 29587 tablet_replica.cc:442] T 5e363998298e41d69f61fb9852c3892d P 4b482e8b109143a28f854c2bee2d815f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 21:12:08.922010 29579 consensus_queue.cc:260] T 5e363998298e41d69f61fb9852c3892d P b45a1e0cc11d4f9c9b52563a245e291b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } }
I20260806 21:12:08.922241 29579 ts_tablet_manager.cc:1428] T 5e363998298e41d69f61fb9852c3892d P b45a1e0cc11d4f9c9b52563a245e291b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 21:12:08.922531 29597 tablet_replica.cc:439] T 5e363998298e41d69f61fb9852c3892d P b45a1e0cc11d4f9c9b52563a245e291b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } } }
I20260806 21:12:08.922672 29597 tablet_replica.cc:442] T 5e363998298e41d69f61fb9852c3892d P b45a1e0cc11d4f9c9b52563a245e291b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 21:12:08.923009 29581 tablet_bootstrap.cc:492] T 5e363998298e41d69f61fb9852c3892d P 18cbdda3a4834b2bbae22c4a599e7d3a: No bootstrap required, opened a new log
I20260806 21:12:08.923089 29581 ts_tablet_manager.cc:1397] T 5e363998298e41d69f61fb9852c3892d P 18cbdda3a4834b2bbae22c4a599e7d3a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 21:12:08.923530 29581 raft_consensus.cc:348] T 5e363998298e41d69f61fb9852c3892d P 18cbdda3a4834b2bbae22c4a599e7d3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } }
I20260806 21:12:08.923641 29581 raft_consensus.cc:374] T 5e363998298e41d69f61fb9852c3892d P 18cbdda3a4834b2bbae22c4a599e7d3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:12:08.923681 29581 raft_consensus.cc:729] T 5e363998298e41d69f61fb9852c3892d P 18cbdda3a4834b2bbae22c4a599e7d3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18cbdda3a4834b2bbae22c4a599e7d3a, State: Initialized, Role: FOLLOWER
I20260806 21:12:08.923803 29581 consensus_queue.cc:260] T 5e363998298e41d69f61fb9852c3892d P 18cbdda3a4834b2bbae22c4a599e7d3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } }
I20260806 21:12:08.924131 29581 ts_tablet_manager.cc:1428] T 5e363998298e41d69f61fb9852c3892d P 18cbdda3a4834b2bbae22c4a599e7d3a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 21:12:08.924250 29585 tablet_replica.cc:439] T 5e363998298e41d69f61fb9852c3892d P 18cbdda3a4834b2bbae22c4a599e7d3a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } } }
I20260806 21:12:08.924423 29585 tablet_replica.cc:442] T 5e363998298e41d69f61fb9852c3892d P 18cbdda3a4834b2bbae22c4a599e7d3a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 21:12:09.081463 29587 raft_consensus.cc:482] T 5e363998298e41d69f61fb9852c3892d P 4b482e8b109143a28f854c2bee2d815f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:12:09.081630 29587 raft_consensus.cc:504] T 5e363998298e41d69f61fb9852c3892d P 4b482e8b109143a28f854c2bee2d815f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } }
I20260806 21:12:09.082814 29587 leader_election.cc:288] T 5e363998298e41d69f61fb9852c3892d P 4b482e8b109143a28f854c2bee2d815f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b45a1e0cc11d4f9c9b52563a245e291b (127.16.178.1:32881), 18cbdda3a4834b2bbae22c4a599e7d3a (127.16.178.2:43593)
I20260806 21:12:09.086697 29576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:09.083260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:49919 (local address 127.16.178.2:43593)
0806 21:12:09.083406 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:09.083410 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:09.083604 (+ 194us) server_negotiation.cc:407] Connection header received
0806 21:12:09.083721 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:09.083725 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:09.083768 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:09.083829 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:12:09.084142 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:09.084720 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:09.085759 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:09.086142 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:09.086250 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:09.086310 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:09.086489 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 21:12:09.086558 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 21:12:09.086720 29645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:09.083171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.2:43593 (local address 127.16.178.3:49919)
0806 21:12:09.083512 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:09.083523 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:09.083633 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:09.083851 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:09.083854 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:09.083905 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:12:09.084023 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:09.084028 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:09.084828 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:09.084831 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:09.085646 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:09.085655 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:09.086250 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:09.086254 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:09.086347 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:09.086398 (+ 51us) client_negotiation.cc:769] Sending connection context
0806 21:12:09.086462 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 21:12:09.086543 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":42,"spinlock_wait_cycles":1664,"thread_start_us":77,"threads_started":1}
I20260806 21:12:09.086714 29572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:09.083200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:60387 (local address 127.16.178.1:32881)
0806 21:12:09.083335 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:09.083339 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:09.083546 (+ 207us) server_negotiation.cc:407] Connection header received
0806 21:12:09.083681 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:09.083684 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:09.083728 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:09.083791 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:12:09.084121 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:09.084632 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:09.085770 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:09.086163 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:09.086273 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:09.086327 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:09.086517 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 21:12:09.086591 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 21:12:09.086750 29644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:09.083125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:32881 (local address 127.16.178.3:60387)
0806 21:12:09.083454 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:09.083467 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:09.083585 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:09.083817 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:09.083820 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:09.083835 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:12:09.083992 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:09.083997 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:09.084744 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:09.084747 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:09.085649 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:09.085659 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:09.086277 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:09.086281 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:09.086401 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:09.086423 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 21:12:09.086486 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 21:12:09.086561 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":40,"spinlock_wait_cycles":51072,"thread_start_us":77,"threads_started":1}
I20260806 21:12:09.087241 29246 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e363998298e41d69f61fb9852c3892d" candidate_uuid: "4b482e8b109143a28f854c2bee2d815f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" is_pre_election: true
I20260806 21:12:09.087273 29007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e363998298e41d69f61fb9852c3892d" candidate_uuid: "4b482e8b109143a28f854c2bee2d815f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" is_pre_election: true
I20260806 21:12:09.087437 29246 raft_consensus.cc:2454] T 5e363998298e41d69f61fb9852c3892d P 18cbdda3a4834b2bbae22c4a599e7d3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b482e8b109143a28f854c2bee2d815f in term 0.
I20260806 21:12:09.087549 29007 raft_consensus.cc:2454] T 5e363998298e41d69f61fb9852c3892d P b45a1e0cc11d4f9c9b52563a245e291b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b482e8b109143a28f854c2bee2d815f in term 0.
I20260806 21:12:09.087872 29361 leader_election.cc:302] T 5e363998298e41d69f61fb9852c3892d P 4b482e8b109143a28f854c2bee2d815f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18cbdda3a4834b2bbae22c4a599e7d3a, 4b482e8b109143a28f854c2bee2d815f; no voters:
I20260806 21:12:09.088217 29587 raft_consensus.cc:2781] T 5e363998298e41d69f61fb9852c3892d P 4b482e8b109143a28f854c2bee2d815f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:12:09.088277 29587 raft_consensus.cc:482] T 5e363998298e41d69f61fb9852c3892d P 4b482e8b109143a28f854c2bee2d815f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:12:09.088312 29587 raft_consensus.cc:3037] T 5e363998298e41d69f61fb9852c3892d P 4b482e8b109143a28f854c2bee2d815f [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:12:09.089751 29587 raft_consensus.cc:504] T 5e363998298e41d69f61fb9852c3892d P 4b482e8b109143a28f854c2bee2d815f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } }
I20260806 21:12:09.090142 29587 leader_election.cc:288] T 5e363998298e41d69f61fb9852c3892d P 4b482e8b109143a28f854c2bee2d815f [CANDIDATE]: Term 1 election: Requested vote from peers b45a1e0cc11d4f9c9b52563a245e291b (127.16.178.1:32881), 18cbdda3a4834b2bbae22c4a599e7d3a (127.16.178.2:43593)
I20260806 21:12:09.090556 29007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e363998298e41d69f61fb9852c3892d" candidate_uuid: "4b482e8b109143a28f854c2bee2d815f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b45a1e0cc11d4f9c9b52563a245e291b"
I20260806 21:12:09.090572 29246 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e363998298e41d69f61fb9852c3892d" candidate_uuid: "4b482e8b109143a28f854c2bee2d815f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a"
I20260806 21:12:09.090683 29007 raft_consensus.cc:3037] T 5e363998298e41d69f61fb9852c3892d P b45a1e0cc11d4f9c9b52563a245e291b [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:12:09.090704 29246 raft_consensus.cc:3037] T 5e363998298e41d69f61fb9852c3892d P 18cbdda3a4834b2bbae22c4a599e7d3a [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:12:09.091769 29246 raft_consensus.cc:2454] T 5e363998298e41d69f61fb9852c3892d P 18cbdda3a4834b2bbae22c4a599e7d3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b482e8b109143a28f854c2bee2d815f in term 1.
I20260806 21:12:09.091866 29007 raft_consensus.cc:2454] T 5e363998298e41d69f61fb9852c3892d P b45a1e0cc11d4f9c9b52563a245e291b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b482e8b109143a28f854c2bee2d815f in term 1.
I20260806 21:12:09.092226 29362 leader_election.cc:302] T 5e363998298e41d69f61fb9852c3892d P 4b482e8b109143a28f854c2bee2d815f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 18cbdda3a4834b2bbae22c4a599e7d3a, 4b482e8b109143a28f854c2bee2d815f, b45a1e0cc11d4f9c9b52563a245e291b; no voters:
I20260806 21:12:09.092442 29587 raft_consensus.cc:2781] T 5e363998298e41d69f61fb9852c3892d P 4b482e8b109143a28f854c2bee2d815f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:12:09.092656 29587 raft_consensus.cc:686] T 5e363998298e41d69f61fb9852c3892d P 4b482e8b109143a28f854c2bee2d815f [term 1 LEADER]: Becoming Leader. State: Replica: 4b482e8b109143a28f854c2bee2d815f, State: Running, Role: LEADER
I20260806 21:12:09.093115 29587 consensus_queue.cc:237] T 5e363998298e41d69f61fb9852c3892d P 4b482e8b109143a28f854c2bee2d815f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } }
I20260806 21:12:09.094360 29646 tablet_replica.cc:439] T 5e363998298e41d69f61fb9852c3892d P 4b482e8b109143a28f854c2bee2d815f: TxnStatusTablet state changed. Reason: New leader 4b482e8b109143a28f854c2bee2d815f. Latest consensus state: current_term: 1 leader_uuid: "4b482e8b109143a28f854c2bee2d815f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } } }
I20260806 21:12:09.094498 29646 tablet_replica.cc:442] T 5e363998298e41d69f61fb9852c3892d P 4b482e8b109143a28f854c2bee2d815f: This TxnStatusTablet replica's current role is: LEADER
I20260806 21:12:09.094717 29651 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 21:12:09.094877 29651 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 21:12:09.095738 28763 catalog_manager.cc:5495] T 5e363998298e41d69f61fb9852c3892d P 4b482e8b109143a28f854c2bee2d815f reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b482e8b109143a28f854c2bee2d815f (127.16.178.3). New cstate: current_term: 1 leader_uuid: "4b482e8b109143a28f854c2bee2d815f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } health_report { overall_health: HEALTHY } } }
I20260806 21:12:09.125687 29636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:09.120650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.3:44397 (local address 127.0.0.1:38954)
0806 21:12:09.120828 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:09.120845 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:09.120982 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:09.121451 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:09.121454 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:09.121464 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 21:12:09.121595 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:09.121603 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:09.122662 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:09.122666 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:09.123705 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:09.123719 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:09.124326 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:09.124330 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:09.124422 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:09.124496 (+ 74us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 21:12:09.125356 (+ 860us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 21:12:09.125368 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 21:12:09.125430 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 21:12:09.125536 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 21:12:09.125689 29574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:09.120967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38954 (local address 127.16.178.3:44397)
0806 21:12:09.121105 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:09.121110 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:09.121127 (+ 17us) server_negotiation.cc:407] Connection header received
0806 21:12:09.121222 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:09.121226 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:09.121300 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:09.121416 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:12:09.121717 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:09.122516 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:09.123833 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:09.124214 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:09.124321 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:09.124663 (+ 342us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:12:09.125262 (+ 599us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:12:09.125379 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:09.125467 (+ 88us) server_negotiation.cc:299] Negotiation successful
0806 21:12:09.125534 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 21:12:09.129127 29246 raft_consensus.cc:1261] T 5e363998298e41d69f61fb9852c3892d P 18cbdda3a4834b2bbae22c4a599e7d3a [term 1 FOLLOWER]: Refusing update from remote peer 4b482e8b109143a28f854c2bee2d815f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:12:09.129350 29007 raft_consensus.cc:1261] T 5e363998298e41d69f61fb9852c3892d P b45a1e0cc11d4f9c9b52563a245e291b [term 1 FOLLOWER]: Refusing update from remote peer 4b482e8b109143a28f854c2bee2d815f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:12:09.129877 29587 consensus_queue.cc:1035] T 5e363998298e41d69f61fb9852c3892d P 4b482e8b109143a28f854c2bee2d815f [LEADER]: Connected to new peer: Peer: permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:12:09.130304 29587 consensus_queue.cc:1035] T 5e363998298e41d69f61fb9852c3892d P 4b482e8b109143a28f854c2bee2d815f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:12:09.133322 29597 tablet_replica.cc:439] T 5e363998298e41d69f61fb9852c3892d P b45a1e0cc11d4f9c9b52563a245e291b: TxnStatusTablet state changed. Reason: New leader 4b482e8b109143a28f854c2bee2d815f. Latest consensus state: current_term: 1 leader_uuid: "4b482e8b109143a28f854c2bee2d815f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } } }
I20260806 21:12:09.133469 29597 tablet_replica.cc:442] T 5e363998298e41d69f61fb9852c3892d P b45a1e0cc11d4f9c9b52563a245e291b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 21:12:09.134624 29646 tablet_replica.cc:439] T 5e363998298e41d69f61fb9852c3892d P 4b482e8b109143a28f854c2bee2d815f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4b482e8b109143a28f854c2bee2d815f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } } }
I20260806 21:12:09.134827 29646 tablet_replica.cc:442] T 5e363998298e41d69f61fb9852c3892d P 4b482e8b109143a28f854c2bee2d815f: This TxnStatusTablet replica's current role is: LEADER
I20260806 21:12:09.134815 29604 mvcc.cc:205] Tried to move back new op lower bound from 7315663786507255808 to 7315663786368135168. Current Snapshot: MvccSnapshot[applied={T|T < 7315663786507255808}]
I20260806 21:12:09.135388 29585 tablet_replica.cc:439] T 5e363998298e41d69f61fb9852c3892d P 18cbdda3a4834b2bbae22c4a599e7d3a: TxnStatusTablet state changed. Reason: New leader 4b482e8b109143a28f854c2bee2d815f. Latest consensus state: current_term: 1 leader_uuid: "4b482e8b109143a28f854c2bee2d815f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } } }
I20260806 21:12:09.135511 29585 tablet_replica.cc:442] T 5e363998298e41d69f61fb9852c3892d P 18cbdda3a4834b2bbae22c4a599e7d3a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 21:12:09.135619 29585 tablet_replica.cc:439] T 5e363998298e41d69f61fb9852c3892d P 18cbdda3a4834b2bbae22c4a599e7d3a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4b482e8b109143a28f854c2bee2d815f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } } }
I20260806 21:12:09.135687 29585 tablet_replica.cc:442] T 5e363998298e41d69f61fb9852c3892d P 18cbdda3a4834b2bbae22c4a599e7d3a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 21:12:09.137852 29587 tablet_replica.cc:439] T 5e363998298e41d69f61fb9852c3892d P 4b482e8b109143a28f854c2bee2d815f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4b482e8b109143a28f854c2bee2d815f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } } }
I20260806 21:12:09.137974 29587 tablet_replica.cc:442] T 5e363998298e41d69f61fb9852c3892d P 4b482e8b109143a28f854c2bee2d815f: This TxnStatusTablet replica's current role is: LEADER
I20260806 21:12:09.139241 29597 tablet_replica.cc:439] T 5e363998298e41d69f61fb9852c3892d P b45a1e0cc11d4f9c9b52563a245e291b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4b482e8b109143a28f854c2bee2d815f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } } }
I20260806 21:12:09.139387 29597 tablet_replica.cc:442] T 5e363998298e41d69f61fb9852c3892d P b45a1e0cc11d4f9c9b52563a245e291b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 21:12:09.151443 29574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:09.147669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:44969 (local address 127.16.178.3:44397)
0806 21:12:09.147799 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:09.147803 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:09.147905 (+ 102us) server_negotiation.cc:407] Connection header received
0806 21:12:09.148056 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:09.148059 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:09.148226 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:09.148313 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:12:09.148624 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:09.149157 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:09.150396 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:09.150618 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:09.150713 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:09.151050 (+ 337us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:12:09.151128 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:12:09.151175 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:09.151275 (+ 100us) server_negotiation.cc:299] Negotiation successful
0806 21:12:09.151321 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 21:12:09.151535 29667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:09.147431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.3:44397 (local address 127.16.178.1:44969)
0806 21:12:09.147819 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:09.147834 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:09.147963 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:09.148364 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:09.148368 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:09.148378 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 21:12:09.148505 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:09.148512 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:09.149256 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:09.149259 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:09.150258 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:09.150270 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:09.150733 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:09.150737 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:09.150832 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:09.150903 (+ 71us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 21:12:09.151196 (+ 293us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 21:12:09.151201 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 21:12:09.151269 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 21:12:09.151362 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":101,"threads_started":1}
I20260806 21:12:09.159227 29246 raft_consensus.cc:1261] T 07e2f1592efc43548b02aa5b31003a92 P 18cbdda3a4834b2bbae22c4a599e7d3a [term 1 FOLLOWER]: Refusing update from remote peer b45a1e0cc11d4f9c9b52563a245e291b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:12:09.159227 29464 raft_consensus.cc:1261] T 07e2f1592efc43548b02aa5b31003a92 P 4b482e8b109143a28f854c2bee2d815f [term 1 FOLLOWER]: Refusing update from remote peer b45a1e0cc11d4f9c9b52563a245e291b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:12:09.159663 29597 consensus_queue.cc:1035] T 07e2f1592efc43548b02aa5b31003a92 P b45a1e0cc11d4f9c9b52563a245e291b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:12:09.159847 29612 consensus_queue.cc:1035] T 07e2f1592efc43548b02aa5b31003a92 P b45a1e0cc11d4f9c9b52563a245e291b [LEADER]: Connected to new peer: Peer: permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:12:09.165227 29603 mvcc.cc:205] Tried to move back new op lower bound from 7315663786632544256 to 7315663785385656320. Current Snapshot: MvccSnapshot[applied={T|T < 7315663786632544256}]
I20260806 21:12:09.184755 29572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:09.180478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:41813 (local address 127.16.178.1:32881)
0806 21:12:09.180591 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:09.180595 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:09.180823 (+ 228us) server_negotiation.cc:407] Connection header received
0806 21:12:09.180994 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:09.180997 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:09.181062 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:09.181125 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:12:09.181515 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:09.182274 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:09.183635 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:09.183851 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:09.183935 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:09.184286 (+ 351us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:12:09.184372 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:12:09.184443 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:09.184555 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 21:12:09.184615 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 21:12:09.185093 29677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:09.180437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:32881 (local address 127.16.178.3:41813)
0806 21:12:09.180748 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:09.180760 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:09.180893 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:09.181240 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:09.181243 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:09.181251 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 21:12:09.181413 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:09.181420 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:09.182378 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:09.182382 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:09.183511 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:09.183524 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:09.183971 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:09.183975 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:09.184073 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:09.184166 (+ 93us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 21:12:09.184454 (+ 288us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 21:12:09.184460 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 21:12:09.184525 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 21:12:09.184606 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":93,"threads_started":1}
I20260806 21:12:09.205515 28763 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39258:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:12:09.205682 28763 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39258:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:12:09.207981 28763 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9f793d6ef89b4cd0a78f73c7c77df3f9: Sending DeleteTablet for 3 replicas of tablet 07e2f1592efc43548b02aa5b31003a92
I20260806 21:12:09.208565 28970 tablet_service.cc:1514] Processing DeleteTablet for tablet 07e2f1592efc43548b02aa5b31003a92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:12:09 UTC) from {username='oryx', principal='oryx/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:47160
I20260806 21:12:09.208751 29618 tablet_replica.cc:330] T 07e2f1592efc43548b02aa5b31003a92 P b45a1e0cc11d4f9c9b52563a245e291b: stopping tablet replica
I20260806 21:12:09.208724 29207 tablet_service.cc:1514] Processing DeleteTablet for tablet 07e2f1592efc43548b02aa5b31003a92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:12:09 UTC) from {username='oryx', principal='oryx/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:54552
I20260806 21:12:09.208868 29618 raft_consensus.cc:2229] T 07e2f1592efc43548b02aa5b31003a92 P b45a1e0cc11d4f9c9b52563a245e291b [term 1 LEADER]: Raft consensus shutting down.
I20260806 21:12:09.208937 29622 tablet_replica.cc:330] T 07e2f1592efc43548b02aa5b31003a92 P 18cbdda3a4834b2bbae22c4a599e7d3a: stopping tablet replica
I20260806 21:12:09.209041 29622 raft_consensus.cc:2229] T 07e2f1592efc43548b02aa5b31003a92 P 18cbdda3a4834b2bbae22c4a599e7d3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:12:09.209082 29618 raft_consensus.cc:2258] T 07e2f1592efc43548b02aa5b31003a92 P b45a1e0cc11d4f9c9b52563a245e291b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:12:09.209151 29622 raft_consensus.cc:2258] T 07e2f1592efc43548b02aa5b31003a92 P 18cbdda3a4834b2bbae22c4a599e7d3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:12:09.209940 29622 ts_tablet_manager.cc:1905] T 07e2f1592efc43548b02aa5b31003a92 P 18cbdda3a4834b2bbae22c4a599e7d3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:12:09.211827 29618 ts_tablet_manager.cc:1905] T 07e2f1592efc43548b02aa5b31003a92 P b45a1e0cc11d4f9c9b52563a245e291b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:12:09.212474 29437 tablet_service.cc:1514] Processing DeleteTablet for tablet 07e2f1592efc43548b02aa5b31003a92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:12:09 UTC) from {username='oryx', principal='oryx/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:38944
I20260806 21:12:09.214193 29618 ts_tablet_manager.cc:1918] T 07e2f1592efc43548b02aa5b31003a92 P b45a1e0cc11d4f9c9b52563a245e291b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 21:12:09.214264 29618 log.cc:1196] T 07e2f1592efc43548b02aa5b31003a92 P b45a1e0cc11d4f9c9b52563a245e291b: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-0/wal/wals/07e2f1592efc43548b02aa5b31003a92
I20260806 21:12:09.214529 29618 ts_tablet_manager.cc:1939] T 07e2f1592efc43548b02aa5b31003a92 P b45a1e0cc11d4f9c9b52563a245e291b: Deleting consensus metadata
I20260806 21:12:09.214529 29620 tablet_replica.cc:330] T 07e2f1592efc43548b02aa5b31003a92 P 4b482e8b109143a28f854c2bee2d815f: stopping tablet replica
I20260806 21:12:09.214648 29620 raft_consensus.cc:2229] T 07e2f1592efc43548b02aa5b31003a92 P 4b482e8b109143a28f854c2bee2d815f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:12:09.214777 29620 raft_consensus.cc:2258] T 07e2f1592efc43548b02aa5b31003a92 P 4b482e8b109143a28f854c2bee2d815f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:12:09.215584 29620 ts_tablet_manager.cc:1905] T 07e2f1592efc43548b02aa5b31003a92 P 4b482e8b109143a28f854c2bee2d815f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:12:09.216995 29622 ts_tablet_manager.cc:1918] T 07e2f1592efc43548b02aa5b31003a92 P 18cbdda3a4834b2bbae22c4a599e7d3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 21:12:09.217109 29622 log.cc:1196] T 07e2f1592efc43548b02aa5b31003a92 P 18cbdda3a4834b2bbae22c4a599e7d3a: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-1/wal/wals/07e2f1592efc43548b02aa5b31003a92
I20260806 21:12:09.217371 29622 ts_tablet_manager.cc:1939] T 07e2f1592efc43548b02aa5b31003a92 P 18cbdda3a4834b2bbae22c4a599e7d3a: Deleting consensus metadata
I20260806 21:12:09.218189 28742 catalog_manager.cc:4841] TS 18cbdda3a4834b2bbae22c4a599e7d3a (127.16.178.2:43593): tablet 07e2f1592efc43548b02aa5b31003a92 (table test-table [id=dad7b49d3e274e1d92fee95562a36a5e]) successfully deleted
I20260806 21:12:09.218453 29620 ts_tablet_manager.cc:1918] T 07e2f1592efc43548b02aa5b31003a92 P 4b482e8b109143a28f854c2bee2d815f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 21:12:09.218533 29620 log.cc:1196] T 07e2f1592efc43548b02aa5b31003a92 P 4b482e8b109143a28f854c2bee2d815f: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-2/wal/wals/07e2f1592efc43548b02aa5b31003a92
I20260806 21:12:09.218743 28743 catalog_manager.cc:4841] TS b45a1e0cc11d4f9c9b52563a245e291b (127.16.178.1:32881): tablet 07e2f1592efc43548b02aa5b31003a92 (table test-table [id=dad7b49d3e274e1d92fee95562a36a5e]) successfully deleted
I20260806 21:12:09.218822 29620 ts_tablet_manager.cc:1939] T 07e2f1592efc43548b02aa5b31003a92 P 4b482e8b109143a28f854c2bee2d815f: Deleting consensus metadata
I20260806 21:12:09.219506 28741 catalog_manager.cc:4841] TS 4b482e8b109143a28f854c2bee2d815f (127.16.178.3:44397): tablet 07e2f1592efc43548b02aa5b31003a92 (table test-table [id=dad7b49d3e274e1d92fee95562a36a5e]) successfully deleted
Aug 06 21:12:09 dist-test-slave-lgr5 krb5kdc[28695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050729, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 21:12:09 dist-test-slave-lgr5 krb5kdc[28695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:09 dist-test-slave-lgr5 krb5kdc[28695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
W20260806 21:12:09.245617 28837 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:09.238813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39274 (local address 127.16.178.62:46619)
0806 21:12:09.238947 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:09.238953 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:09.239266 (+ 313us) server_negotiation.cc:407] Connection header received
0806 21:12:09.239455 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:09.239458 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:09.239526 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:09.239595 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:09.240499 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:09.241049 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:09.241949 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:09.242187 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:09.242283 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:09.245477 (+ 3194us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39274: BlockingRecv error: recv got EOF from 127.0.0.1:39274 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 06 21:12:09 dist-test-slave-lgr5 krb5kdc[28695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050729, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM
I20260806 21:12:09.255882 28837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:09.247340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39290 (local address 127.16.178.62:46619)
0806 21:12:09.247544 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:09.247548 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:09.247563 (+ 15us) server_negotiation.cc:407] Connection header received
0806 21:12:09.247695 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:09.247698 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:09.247746 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:09.247810 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:09.248492 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:09.249007 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:09.249785 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:09.250013 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:09.250102 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:09.252654 (+ 2552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:09.252657 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:09.252659 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:09.252692 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:09.254115 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:09.254579 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:09.254582 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:09.254583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:09.254672 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:09.254942 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:09.254946 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:09.254948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:09.255168 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:09.255295 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:09.255572 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 21:12:09.255708 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122}
I20260806 21:12:09.259130 28763 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39290:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:12:09.259640 28763 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:12:09.264940 29207 tablet_service.cc:1467] Processing CreateTablet for tablet 655d05c3a3914910a4db7b19a700e4b3 (DEFAULT_TABLE table=test-table [id=1f3270e7023b46948176af16bb0590ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:12:09.264919 28970 tablet_service.cc:1467] Processing CreateTablet for tablet 655d05c3a3914910a4db7b19a700e4b3 (DEFAULT_TABLE table=test-table [id=1f3270e7023b46948176af16bb0590ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:12:09.265100 29437 tablet_service.cc:1467] Processing CreateTablet for tablet 655d05c3a3914910a4db7b19a700e4b3 (DEFAULT_TABLE table=test-table [id=1f3270e7023b46948176af16bb0590ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:12:09.265260 29207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 655d05c3a3914910a4db7b19a700e4b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:12:09.265268 28970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 655d05c3a3914910a4db7b19a700e4b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:12:09.265362 29437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 655d05c3a3914910a4db7b19a700e4b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:12:09.267798 29581 tablet_bootstrap.cc:492] T 655d05c3a3914910a4db7b19a700e4b3 P 18cbdda3a4834b2bbae22c4a599e7d3a: Bootstrap starting.
I20260806 21:12:09.267911 29580 tablet_bootstrap.cc:492] T 655d05c3a3914910a4db7b19a700e4b3 P 4b482e8b109143a28f854c2bee2d815f: Bootstrap starting.
I20260806 21:12:09.268983 29580 tablet_bootstrap.cc:654] T 655d05c3a3914910a4db7b19a700e4b3 P 4b482e8b109143a28f854c2bee2d815f: Neither blocks nor log segments found. Creating new log.
I20260806 21:12:09.268999 29581 tablet_bootstrap.cc:654] T 655d05c3a3914910a4db7b19a700e4b3 P 18cbdda3a4834b2bbae22c4a599e7d3a: Neither blocks nor log segments found. Creating new log.
I20260806 21:12:09.269734 29579 tablet_bootstrap.cc:492] T 655d05c3a3914910a4db7b19a700e4b3 P b45a1e0cc11d4f9c9b52563a245e291b: Bootstrap starting.
I20260806 21:12:09.270517 29580 tablet_bootstrap.cc:492] T 655d05c3a3914910a4db7b19a700e4b3 P 4b482e8b109143a28f854c2bee2d815f: No bootstrap required, opened a new log
I20260806 21:12:09.270630 29579 tablet_bootstrap.cc:654] T 655d05c3a3914910a4db7b19a700e4b3 P b45a1e0cc11d4f9c9b52563a245e291b: Neither blocks nor log segments found. Creating new log.
I20260806 21:12:09.270632 29580 ts_tablet_manager.cc:1397] T 655d05c3a3914910a4db7b19a700e4b3 P 4b482e8b109143a28f854c2bee2d815f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 21:12:09.271075 29580 raft_consensus.cc:348] T 655d05c3a3914910a4db7b19a700e4b3 P 4b482e8b109143a28f854c2bee2d815f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } }
I20260806 21:12:09.271230 29580 raft_consensus.cc:374] T 655d05c3a3914910a4db7b19a700e4b3 P 4b482e8b109143a28f854c2bee2d815f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:12:09.271277 29580 raft_consensus.cc:729] T 655d05c3a3914910a4db7b19a700e4b3 P 4b482e8b109143a28f854c2bee2d815f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b482e8b109143a28f854c2bee2d815f, State: Initialized, Role: FOLLOWER
I20260806 21:12:09.271427 29580 consensus_queue.cc:260] T 655d05c3a3914910a4db7b19a700e4b3 P 4b482e8b109143a28f854c2bee2d815f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } }
I20260806 21:12:09.272394 29579 tablet_bootstrap.cc:492] T 655d05c3a3914910a4db7b19a700e4b3 P b45a1e0cc11d4f9c9b52563a245e291b: No bootstrap required, opened a new log
I20260806 21:12:09.272508 29579 ts_tablet_manager.cc:1397] T 655d05c3a3914910a4db7b19a700e4b3 P b45a1e0cc11d4f9c9b52563a245e291b: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 21:12:09.272965 29579 raft_consensus.cc:348] T 655d05c3a3914910a4db7b19a700e4b3 P b45a1e0cc11d4f9c9b52563a245e291b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } }
I20260806 21:12:09.273108 29579 raft_consensus.cc:374] T 655d05c3a3914910a4db7b19a700e4b3 P b45a1e0cc11d4f9c9b52563a245e291b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:12:09.273141 29579 raft_consensus.cc:729] T 655d05c3a3914910a4db7b19a700e4b3 P b45a1e0cc11d4f9c9b52563a245e291b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b45a1e0cc11d4f9c9b52563a245e291b, State: Initialized, Role: FOLLOWER
I20260806 21:12:09.273285 29579 consensus_queue.cc:260] T 655d05c3a3914910a4db7b19a700e4b3 P b45a1e0cc11d4f9c9b52563a245e291b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } }
I20260806 21:12:09.273509 29580 ts_tablet_manager.cc:1428] T 655d05c3a3914910a4db7b19a700e4b3 P 4b482e8b109143a28f854c2bee2d815f: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260806 21:12:09.273944 29581 tablet_bootstrap.cc:492] T 655d05c3a3914910a4db7b19a700e4b3 P 18cbdda3a4834b2bbae22c4a599e7d3a: No bootstrap required, opened a new log
I20260806 21:12:09.274065 29581 ts_tablet_manager.cc:1397] T 655d05c3a3914910a4db7b19a700e4b3 P 18cbdda3a4834b2bbae22c4a599e7d3a: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260806 21:12:09.274426 29579 ts_tablet_manager.cc:1428] T 655d05c3a3914910a4db7b19a700e4b3 P b45a1e0cc11d4f9c9b52563a245e291b: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260806 21:12:09.274601 29581 raft_consensus.cc:348] T 655d05c3a3914910a4db7b19a700e4b3 P 18cbdda3a4834b2bbae22c4a599e7d3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } }
I20260806 21:12:09.274729 29581 raft_consensus.cc:374] T 655d05c3a3914910a4db7b19a700e4b3 P 18cbdda3a4834b2bbae22c4a599e7d3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:12:09.274760 29581 raft_consensus.cc:729] T 655d05c3a3914910a4db7b19a700e4b3 P 18cbdda3a4834b2bbae22c4a599e7d3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18cbdda3a4834b2bbae22c4a599e7d3a, State: Initialized, Role: FOLLOWER
I20260806 21:12:09.274889 29581 consensus_queue.cc:260] T 655d05c3a3914910a4db7b19a700e4b3 P 18cbdda3a4834b2bbae22c4a599e7d3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } }
I20260806 21:12:09.275677 29581 ts_tablet_manager.cc:1428] T 655d05c3a3914910a4db7b19a700e4b3 P 18cbdda3a4834b2bbae22c4a599e7d3a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 21:12:09.389716 29661 raft_consensus.cc:482] T 655d05c3a3914910a4db7b19a700e4b3 P 18cbdda3a4834b2bbae22c4a599e7d3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:12:09.389885 29661 raft_consensus.cc:504] T 655d05c3a3914910a4db7b19a700e4b3 P 18cbdda3a4834b2bbae22c4a599e7d3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } }
I20260806 21:12:09.391067 29661 leader_election.cc:288] T 655d05c3a3914910a4db7b19a700e4b3 P 18cbdda3a4834b2bbae22c4a599e7d3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b45a1e0cc11d4f9c9b52563a245e291b (127.16.178.1:32881), 4b482e8b109143a28f854c2bee2d815f (127.16.178.3:44397)
I20260806 21:12:09.395179 29574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:09.391577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:42561 (local address 127.16.178.3:44397)
0806 21:12:09.391684 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:09.391688 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:09.391929 (+ 241us) server_negotiation.cc:407] Connection header received
0806 21:12:09.392131 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:09.392134 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:09.392182 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:09.392246 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:12:09.392536 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:09.393065 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:09.394244 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:09.394638 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:09.394748 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:09.394808 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:09.394972 (+ 164us) server_negotiation.cc:299] Negotiation successful
0806 21:12:09.395044 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 21:12:09.395236 29708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:09.391345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.3:44397 (local address 127.16.178.2:42561)
0806 21:12:09.391832 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:09.391844 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:09.391989 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:09.392277 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:09.392280 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:09.392296 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:12:09.392433 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:09.392439 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:09.393195 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:09.393198 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:09.394116 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:09.394127 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:09.394753 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:09.394756 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:09.394839 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:09.394874 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 21:12:09.394942 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 21:12:09.395018 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"mutex_wait_us":12,"spinlock_wait_cycles":15744,"thread_start_us":180,"threads_started":1}
I20260806 21:12:09.395241 29572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:09.391442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:39599 (local address 127.16.178.1:32881)
0806 21:12:09.391591 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:09.391596 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:09.391938 (+ 342us) server_negotiation.cc:407] Connection header received
0806 21:12:09.392126 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:09.392131 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:09.392183 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:09.392258 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:12:09.392607 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:09.393119 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:09.394263 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:09.394667 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:09.394790 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:09.394853 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:09.395017 (+ 164us) server_negotiation.cc:299] Negotiation successful
0806 21:12:09.395103 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 21:12:09.395491 29707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:09.391324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:32881 (local address 127.16.178.2:39599)
0806 21:12:09.391799 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:09.391815 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:09.391990 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:09.392318 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:09.392321 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:09.392340 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:12:09.392480 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:09.392486 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:09.393233 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:09.393237 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:09.394122 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:09.394133 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:09.394787 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:09.394791 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:09.394886 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:09.394909 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 21:12:09.394978 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 21:12:09.395065 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"mutex_wait_us":39,"spinlock_wait_cycles":28928,"thread_start_us":102,"threads_started":1}
I20260806 21:12:09.395957 29007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "655d05c3a3914910a4db7b19a700e4b3" candidate_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" is_pre_election: true
I20260806 21:12:09.395957 29464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "655d05c3a3914910a4db7b19a700e4b3" candidate_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b482e8b109143a28f854c2bee2d815f" is_pre_election: true
I20260806 21:12:09.396164 29007 raft_consensus.cc:2454] T 655d05c3a3914910a4db7b19a700e4b3 P b45a1e0cc11d4f9c9b52563a245e291b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18cbdda3a4834b2bbae22c4a599e7d3a in term 0.
I20260806 21:12:09.396164 29464 raft_consensus.cc:2454] T 655d05c3a3914910a4db7b19a700e4b3 P 4b482e8b109143a28f854c2bee2d815f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18cbdda3a4834b2bbae22c4a599e7d3a in term 0.
I20260806 21:12:09.396626 29121 leader_election.cc:302] T 655d05c3a3914910a4db7b19a700e4b3 P 18cbdda3a4834b2bbae22c4a599e7d3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18cbdda3a4834b2bbae22c4a599e7d3a, b45a1e0cc11d4f9c9b52563a245e291b; no voters:
I20260806 21:12:09.396942 29661 raft_consensus.cc:2781] T 655d05c3a3914910a4db7b19a700e4b3 P 18cbdda3a4834b2bbae22c4a599e7d3a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:12:09.397009 29661 raft_consensus.cc:482] T 655d05c3a3914910a4db7b19a700e4b3 P 18cbdda3a4834b2bbae22c4a599e7d3a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:12:09.397054 29661 raft_consensus.cc:3037] T 655d05c3a3914910a4db7b19a700e4b3 P 18cbdda3a4834b2bbae22c4a599e7d3a [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:12:09.398167 29661 raft_consensus.cc:504] T 655d05c3a3914910a4db7b19a700e4b3 P 18cbdda3a4834b2bbae22c4a599e7d3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } }
I20260806 21:12:09.398562 29661 leader_election.cc:288] T 655d05c3a3914910a4db7b19a700e4b3 P 18cbdda3a4834b2bbae22c4a599e7d3a [CANDIDATE]: Term 1 election: Requested vote from peers b45a1e0cc11d4f9c9b52563a245e291b (127.16.178.1:32881), 4b482e8b109143a28f854c2bee2d815f (127.16.178.3:44397)
I20260806 21:12:09.399010 29464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "655d05c3a3914910a4db7b19a700e4b3" candidate_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b482e8b109143a28f854c2bee2d815f"
I20260806 21:12:09.398999 29007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "655d05c3a3914910a4db7b19a700e4b3" candidate_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b45a1e0cc11d4f9c9b52563a245e291b"
I20260806 21:12:09.399156 29007 raft_consensus.cc:3037] T 655d05c3a3914910a4db7b19a700e4b3 P b45a1e0cc11d4f9c9b52563a245e291b [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:12:09.399156 29464 raft_consensus.cc:3037] T 655d05c3a3914910a4db7b19a700e4b3 P 4b482e8b109143a28f854c2bee2d815f [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:12:09.400444 29007 raft_consensus.cc:2454] T 655d05c3a3914910a4db7b19a700e4b3 P b45a1e0cc11d4f9c9b52563a245e291b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18cbdda3a4834b2bbae22c4a599e7d3a in term 1.
I20260806 21:12:09.400444 29464 raft_consensus.cc:2454] T 655d05c3a3914910a4db7b19a700e4b3 P 4b482e8b109143a28f854c2bee2d815f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18cbdda3a4834b2bbae22c4a599e7d3a in term 1.
I20260806 21:12:09.400779 29121 leader_election.cc:302] T 655d05c3a3914910a4db7b19a700e4b3 P 18cbdda3a4834b2bbae22c4a599e7d3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18cbdda3a4834b2bbae22c4a599e7d3a, b45a1e0cc11d4f9c9b52563a245e291b; no voters:
I20260806 21:12:09.400967 29661 raft_consensus.cc:2781] T 655d05c3a3914910a4db7b19a700e4b3 P 18cbdda3a4834b2bbae22c4a599e7d3a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:12:09.401081 29661 raft_consensus.cc:686] T 655d05c3a3914910a4db7b19a700e4b3 P 18cbdda3a4834b2bbae22c4a599e7d3a [term 1 LEADER]: Becoming Leader. State: Replica: 18cbdda3a4834b2bbae22c4a599e7d3a, State: Running, Role: LEADER
I20260806 21:12:09.401347 29661 consensus_queue.cc:237] T 655d05c3a3914910a4db7b19a700e4b3 P 18cbdda3a4834b2bbae22c4a599e7d3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } }
I20260806 21:12:09.403908 28767 catalog_manager.cc:5495] T 655d05c3a3914910a4db7b19a700e4b3 P 18cbdda3a4834b2bbae22c4a599e7d3a reported cstate change: term changed from 0 to 1, leader changed from <none> to 18cbdda3a4834b2bbae22c4a599e7d3a (127.16.178.2). New cstate: current_term: 1 leader_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } health_report { overall_health: UNKNOWN } } }
I20260806 21:12:09.434098 29576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:09.430581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54566 (local address 127.16.178.2:43593)
0806 21:12:09.430713 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:09.430717 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:09.430733 (+ 16us) server_negotiation.cc:407] Connection header received
0806 21:12:09.430842 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:09.430844 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:09.430902 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:09.431000 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:12:09.431232 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:09.431758 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:09.432659 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:09.432874 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:09.432962 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:09.433220 (+ 258us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:12:09.433704 (+ 484us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:12:09.433809 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:09.433895 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 21:12:09.433950 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 21:12:09.436796 29464 raft_consensus.cc:1261] T 655d05c3a3914910a4db7b19a700e4b3 P 4b482e8b109143a28f854c2bee2d815f [term 1 FOLLOWER]: Refusing update from remote peer 18cbdda3a4834b2bbae22c4a599e7d3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:12:09.436825 29007 raft_consensus.cc:1261] T 655d05c3a3914910a4db7b19a700e4b3 P b45a1e0cc11d4f9c9b52563a245e291b [term 1 FOLLOWER]: Refusing update from remote peer 18cbdda3a4834b2bbae22c4a599e7d3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:12:09.437358 29661 consensus_queue.cc:1035] T 655d05c3a3914910a4db7b19a700e4b3 P 18cbdda3a4834b2bbae22c4a599e7d3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:12:09.437557 29585 consensus_queue.cc:1035] T 655d05c3a3914910a4db7b19a700e4b3 P 18cbdda3a4834b2bbae22c4a599e7d3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:12:09.449591 28767 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39290:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:12:09.449790 28767 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39290:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:12:09.452157 28767 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9f793d6ef89b4cd0a78f73c7c77df3f9: Sending DeleteTablet for 3 replicas of tablet 655d05c3a3914910a4db7b19a700e4b3
I20260806 21:12:09.452917 28970 tablet_service.cc:1514] Processing DeleteTablet for tablet 655d05c3a3914910a4db7b19a700e4b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:12:09 UTC) from {username='oryx', principal='oryx/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:47160
I20260806 21:12:09.452931 29437 tablet_service.cc:1514] Processing DeleteTablet for tablet 655d05c3a3914910a4db7b19a700e4b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:12:09 UTC) from {username='oryx', principal='oryx/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:38944
I20260806 21:12:09.453145 29618 tablet_replica.cc:330] T 655d05c3a3914910a4db7b19a700e4b3 P b45a1e0cc11d4f9c9b52563a245e291b: stopping tablet replica
I20260806 21:12:09.453157 29620 tablet_replica.cc:330] T 655d05c3a3914910a4db7b19a700e4b3 P 4b482e8b109143a28f854c2bee2d815f: stopping tablet replica
I20260806 21:12:09.453286 29620 raft_consensus.cc:2229] T 655d05c3a3914910a4db7b19a700e4b3 P 4b482e8b109143a28f854c2bee2d815f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:12:09.453306 29618 raft_consensus.cc:2229] T 655d05c3a3914910a4db7b19a700e4b3 P b45a1e0cc11d4f9c9b52563a245e291b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:12:09.453464 29618 raft_consensus.cc:2258] T 655d05c3a3914910a4db7b19a700e4b3 P b45a1e0cc11d4f9c9b52563a245e291b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:12:09.453480 29620 raft_consensus.cc:2258] T 655d05c3a3914910a4db7b19a700e4b3 P 4b482e8b109143a28f854c2bee2d815f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:12:09.454175 29207 tablet_service.cc:1514] Processing DeleteTablet for tablet 655d05c3a3914910a4db7b19a700e4b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:12:09 UTC) from {username='oryx', principal='oryx/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:54552
I20260806 21:12:09.454396 29622 tablet_replica.cc:330] T 655d05c3a3914910a4db7b19a700e4b3 P 18cbdda3a4834b2bbae22c4a599e7d3a: stopping tablet replica
I20260806 21:12:09.454532 29622 raft_consensus.cc:2229] T 655d05c3a3914910a4db7b19a700e4b3 P 18cbdda3a4834b2bbae22c4a599e7d3a [term 1 LEADER]: Raft consensus shutting down.
I20260806 21:12:09.454638 29620 ts_tablet_manager.cc:1905] T 655d05c3a3914910a4db7b19a700e4b3 P 4b482e8b109143a28f854c2bee2d815f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:12:09.454855 29618 ts_tablet_manager.cc:1905] T 655d05c3a3914910a4db7b19a700e4b3 P b45a1e0cc11d4f9c9b52563a245e291b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:12:09.454924 29622 raft_consensus.cc:2258] T 655d05c3a3914910a4db7b19a700e4b3 P 18cbdda3a4834b2bbae22c4a599e7d3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:12:09.455646 29622 ts_tablet_manager.cc:1905] T 655d05c3a3914910a4db7b19a700e4b3 P 18cbdda3a4834b2bbae22c4a599e7d3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:12:09.457671 29618 ts_tablet_manager.cc:1918] T 655d05c3a3914910a4db7b19a700e4b3 P b45a1e0cc11d4f9c9b52563a245e291b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:12:09.457720 29620 ts_tablet_manager.cc:1918] T 655d05c3a3914910a4db7b19a700e4b3 P 4b482e8b109143a28f854c2bee2d815f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:12:09.457770 29618 log.cc:1196] T 655d05c3a3914910a4db7b19a700e4b3 P b45a1e0cc11d4f9c9b52563a245e291b: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-0/wal/wals/655d05c3a3914910a4db7b19a700e4b3
I20260806 21:12:09.457810 29620 log.cc:1196] T 655d05c3a3914910a4db7b19a700e4b3 P 4b482e8b109143a28f854c2bee2d815f: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-2/wal/wals/655d05c3a3914910a4db7b19a700e4b3
I20260806 21:12:09.458045 29618 ts_tablet_manager.cc:1939] T 655d05c3a3914910a4db7b19a700e4b3 P b45a1e0cc11d4f9c9b52563a245e291b: Deleting consensus metadata
I20260806 21:12:09.458091 29620 ts_tablet_manager.cc:1939] T 655d05c3a3914910a4db7b19a700e4b3 P 4b482e8b109143a28f854c2bee2d815f: Deleting consensus metadata
I20260806 21:12:09.458550 29622 ts_tablet_manager.cc:1918] T 655d05c3a3914910a4db7b19a700e4b3 P 18cbdda3a4834b2bbae22c4a599e7d3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:12:09.458639 29622 log.cc:1196] T 655d05c3a3914910a4db7b19a700e4b3 P 18cbdda3a4834b2bbae22c4a599e7d3a: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-1/wal/wals/655d05c3a3914910a4db7b19a700e4b3
I20260806 21:12:09.458876 29622 ts_tablet_manager.cc:1939] T 655d05c3a3914910a4db7b19a700e4b3 P 18cbdda3a4834b2bbae22c4a599e7d3a: Deleting consensus metadata
I20260806 21:12:09.459136 28741 catalog_manager.cc:4841] TS 4b482e8b109143a28f854c2bee2d815f (127.16.178.3:44397): tablet 655d05c3a3914910a4db7b19a700e4b3 (table test-table [id=1f3270e7023b46948176af16bb0590ab]) successfully deleted
I20260806 21:12:09.459635 28742 catalog_manager.cc:4841] TS 18cbdda3a4834b2bbae22c4a599e7d3a (127.16.178.2:43593): tablet 655d05c3a3914910a4db7b19a700e4b3 (table test-table [id=1f3270e7023b46948176af16bb0590ab]) successfully deleted
I20260806 21:12:09.459635 28743 catalog_manager.cc:4841] TS b45a1e0cc11d4f9c9b52563a245e291b (127.16.178.1:32881): tablet 655d05c3a3914910a4db7b19a700e4b3 (table test-table [id=1f3270e7023b46948176af16bb0590ab]) successfully deleted
I20260806 21:12:09.460527 28767 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39290:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:12:09.461084 28767 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:12:09.466413 28970 tablet_service.cc:1467] Processing CreateTablet for tablet 93054c2dd4de4689b0b200f97934a499 (DEFAULT_TABLE table=test-table [id=962c786da6bc4630a830e939fdab01eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:12:09.466547 29207 tablet_service.cc:1467] Processing CreateTablet for tablet 93054c2dd4de4689b0b200f97934a499 (DEFAULT_TABLE table=test-table [id=962c786da6bc4630a830e939fdab01eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:12:09.466701 29437 tablet_service.cc:1467] Processing CreateTablet for tablet 93054c2dd4de4689b0b200f97934a499 (DEFAULT_TABLE table=test-table [id=962c786da6bc4630a830e939fdab01eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:12:09.466787 29207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93054c2dd4de4689b0b200f97934a499. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:12:09.466789 28970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93054c2dd4de4689b0b200f97934a499. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:12:09.467029 29437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93054c2dd4de4689b0b200f97934a499. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:12:09.469605 29579 tablet_bootstrap.cc:492] T 93054c2dd4de4689b0b200f97934a499 P b45a1e0cc11d4f9c9b52563a245e291b: Bootstrap starting.
I20260806 21:12:09.469813 29581 tablet_bootstrap.cc:492] T 93054c2dd4de4689b0b200f97934a499 P 18cbdda3a4834b2bbae22c4a599e7d3a: Bootstrap starting.
I20260806 21:12:09.470546 29579 tablet_bootstrap.cc:654] T 93054c2dd4de4689b0b200f97934a499 P b45a1e0cc11d4f9c9b52563a245e291b: Neither blocks nor log segments found. Creating new log.
I20260806 21:12:09.470620 29581 tablet_bootstrap.cc:654] T 93054c2dd4de4689b0b200f97934a499 P 18cbdda3a4834b2bbae22c4a599e7d3a: Neither blocks nor log segments found. Creating new log.
I20260806 21:12:09.471949 29579 tablet_bootstrap.cc:492] T 93054c2dd4de4689b0b200f97934a499 P b45a1e0cc11d4f9c9b52563a245e291b: No bootstrap required, opened a new log
I20260806 21:12:09.472054 29579 ts_tablet_manager.cc:1397] T 93054c2dd4de4689b0b200f97934a499 P b45a1e0cc11d4f9c9b52563a245e291b: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260806 21:12:09.472280 29581 tablet_bootstrap.cc:492] T 93054c2dd4de4689b0b200f97934a499 P 18cbdda3a4834b2bbae22c4a599e7d3a: No bootstrap required, opened a new log
I20260806 21:12:09.472371 29581 ts_tablet_manager.cc:1397] T 93054c2dd4de4689b0b200f97934a499 P 18cbdda3a4834b2bbae22c4a599e7d3a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 21:12:09.472537 29579 raft_consensus.cc:348] T 93054c2dd4de4689b0b200f97934a499 P b45a1e0cc11d4f9c9b52563a245e291b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } }
I20260806 21:12:09.472656 29579 raft_consensus.cc:374] T 93054c2dd4de4689b0b200f97934a499 P b45a1e0cc11d4f9c9b52563a245e291b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:12:09.472687 29579 raft_consensus.cc:729] T 93054c2dd4de4689b0b200f97934a499 P b45a1e0cc11d4f9c9b52563a245e291b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b45a1e0cc11d4f9c9b52563a245e291b, State: Initialized, Role: FOLLOWER
I20260806 21:12:09.472810 29579 consensus_queue.cc:260] T 93054c2dd4de4689b0b200f97934a499 P b45a1e0cc11d4f9c9b52563a245e291b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } }
I20260806 21:12:09.472862 29581 raft_consensus.cc:348] T 93054c2dd4de4689b0b200f97934a499 P 18cbdda3a4834b2bbae22c4a599e7d3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } }
I20260806 21:12:09.472975 29581 raft_consensus.cc:374] T 93054c2dd4de4689b0b200f97934a499 P 18cbdda3a4834b2bbae22c4a599e7d3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:12:09.473006 29581 raft_consensus.cc:729] T 93054c2dd4de4689b0b200f97934a499 P 18cbdda3a4834b2bbae22c4a599e7d3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18cbdda3a4834b2bbae22c4a599e7d3a, State: Initialized, Role: FOLLOWER
I20260806 21:12:09.473061 29579 ts_tablet_manager.cc:1428] T 93054c2dd4de4689b0b200f97934a499 P b45a1e0cc11d4f9c9b52563a245e291b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 21:12:09.473145 29581 consensus_queue.cc:260] T 93054c2dd4de4689b0b200f97934a499 P 18cbdda3a4834b2bbae22c4a599e7d3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } }
I20260806 21:12:09.473439 29581 ts_tablet_manager.cc:1428] T 93054c2dd4de4689b0b200f97934a499 P 18cbdda3a4834b2bbae22c4a599e7d3a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 21:12:09.473876 29580 tablet_bootstrap.cc:492] T 93054c2dd4de4689b0b200f97934a499 P 4b482e8b109143a28f854c2bee2d815f: Bootstrap starting.
I20260806 21:12:09.474778 29580 tablet_bootstrap.cc:654] T 93054c2dd4de4689b0b200f97934a499 P 4b482e8b109143a28f854c2bee2d815f: Neither blocks nor log segments found. Creating new log.
I20260806 21:12:09.476845 29580 tablet_bootstrap.cc:492] T 93054c2dd4de4689b0b200f97934a499 P 4b482e8b109143a28f854c2bee2d815f: No bootstrap required, opened a new log
I20260806 21:12:09.476953 29580 ts_tablet_manager.cc:1397] T 93054c2dd4de4689b0b200f97934a499 P 4b482e8b109143a28f854c2bee2d815f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 21:12:09.477387 29580 raft_consensus.cc:348] T 93054c2dd4de4689b0b200f97934a499 P 4b482e8b109143a28f854c2bee2d815f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } }
I20260806 21:12:09.477530 29580 raft_consensus.cc:374] T 93054c2dd4de4689b0b200f97934a499 P 4b482e8b109143a28f854c2bee2d815f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:12:09.477568 29580 raft_consensus.cc:729] T 93054c2dd4de4689b0b200f97934a499 P 4b482e8b109143a28f854c2bee2d815f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b482e8b109143a28f854c2bee2d815f, State: Initialized, Role: FOLLOWER
I20260806 21:12:09.477699 29580 consensus_queue.cc:260] T 93054c2dd4de4689b0b200f97934a499 P 4b482e8b109143a28f854c2bee2d815f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } }
I20260806 21:12:09.477969 29580 ts_tablet_manager.cc:1428] T 93054c2dd4de4689b0b200f97934a499 P 4b482e8b109143a28f854c2bee2d815f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 21:12:09.550908 29597 raft_consensus.cc:482] T 93054c2dd4de4689b0b200f97934a499 P b45a1e0cc11d4f9c9b52563a245e291b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:12:09.551069 29597 raft_consensus.cc:504] T 93054c2dd4de4689b0b200f97934a499 P b45a1e0cc11d4f9c9b52563a245e291b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } }
I20260806 21:12:09.551545 29597 leader_election.cc:288] T 93054c2dd4de4689b0b200f97934a499 P b45a1e0cc11d4f9c9b52563a245e291b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18cbdda3a4834b2bbae22c4a599e7d3a (127.16.178.2:43593), 4b482e8b109143a28f854c2bee2d815f (127.16.178.3:44397)
I20260806 21:12:09.552198 29246 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93054c2dd4de4689b0b200f97934a499" candidate_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" is_pre_election: true
I20260806 21:12:09.552213 29464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93054c2dd4de4689b0b200f97934a499" candidate_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b482e8b109143a28f854c2bee2d815f" is_pre_election: true
I20260806 21:12:09.552348 29246 raft_consensus.cc:2454] T 93054c2dd4de4689b0b200f97934a499 P 18cbdda3a4834b2bbae22c4a599e7d3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b45a1e0cc11d4f9c9b52563a245e291b in term 0.
I20260806 21:12:09.552383 29464 raft_consensus.cc:2454] T 93054c2dd4de4689b0b200f97934a499 P 4b482e8b109143a28f854c2bee2d815f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b45a1e0cc11d4f9c9b52563a245e291b in term 0.
I20260806 21:12:09.552711 28898 leader_election.cc:302] T 93054c2dd4de4689b0b200f97934a499 P b45a1e0cc11d4f9c9b52563a245e291b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18cbdda3a4834b2bbae22c4a599e7d3a, b45a1e0cc11d4f9c9b52563a245e291b; no voters:
I20260806 21:12:09.553001 29597 raft_consensus.cc:2781] T 93054c2dd4de4689b0b200f97934a499 P b45a1e0cc11d4f9c9b52563a245e291b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:12:09.553066 29597 raft_consensus.cc:482] T 93054c2dd4de4689b0b200f97934a499 P b45a1e0cc11d4f9c9b52563a245e291b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:12:09.553099 29597 raft_consensus.cc:3037] T 93054c2dd4de4689b0b200f97934a499 P b45a1e0cc11d4f9c9b52563a245e291b [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:12:09.554173 29597 raft_consensus.cc:504] T 93054c2dd4de4689b0b200f97934a499 P b45a1e0cc11d4f9c9b52563a245e291b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } }
I20260806 21:12:09.554522 29597 leader_election.cc:288] T 93054c2dd4de4689b0b200f97934a499 P b45a1e0cc11d4f9c9b52563a245e291b [CANDIDATE]: Term 1 election: Requested vote from peers 18cbdda3a4834b2bbae22c4a599e7d3a (127.16.178.2:43593), 4b482e8b109143a28f854c2bee2d815f (127.16.178.3:44397)
I20260806 21:12:09.554860 29246 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93054c2dd4de4689b0b200f97934a499" candidate_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a"
I20260806 21:12:09.554899 29464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93054c2dd4de4689b0b200f97934a499" candidate_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b482e8b109143a28f854c2bee2d815f"
I20260806 21:12:09.554991 29246 raft_consensus.cc:3037] T 93054c2dd4de4689b0b200f97934a499 P 18cbdda3a4834b2bbae22c4a599e7d3a [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:12:09.555004 29464 raft_consensus.cc:3037] T 93054c2dd4de4689b0b200f97934a499 P 4b482e8b109143a28f854c2bee2d815f [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:12:09.556149 29464 raft_consensus.cc:2454] T 93054c2dd4de4689b0b200f97934a499 P 4b482e8b109143a28f854c2bee2d815f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b45a1e0cc11d4f9c9b52563a245e291b in term 1.
I20260806 21:12:09.556147 29246 raft_consensus.cc:2454] T 93054c2dd4de4689b0b200f97934a499 P 18cbdda3a4834b2bbae22c4a599e7d3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b45a1e0cc11d4f9c9b52563a245e291b in term 1.
I20260806 21:12:09.556452 28896 leader_election.cc:302] T 93054c2dd4de4689b0b200f97934a499 P b45a1e0cc11d4f9c9b52563a245e291b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b482e8b109143a28f854c2bee2d815f, b45a1e0cc11d4f9c9b52563a245e291b; no voters:
I20260806 21:12:09.556679 29597 raft_consensus.cc:2781] T 93054c2dd4de4689b0b200f97934a499 P b45a1e0cc11d4f9c9b52563a245e291b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:12:09.556751 29597 raft_consensus.cc:686] T 93054c2dd4de4689b0b200f97934a499 P b45a1e0cc11d4f9c9b52563a245e291b [term 1 LEADER]: Becoming Leader. State: Replica: b45a1e0cc11d4f9c9b52563a245e291b, State: Running, Role: LEADER
I20260806 21:12:09.556885 29597 consensus_queue.cc:237] T 93054c2dd4de4689b0b200f97934a499 P b45a1e0cc11d4f9c9b52563a245e291b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } }
I20260806 21:12:09.558638 28763 catalog_manager.cc:5495] T 93054c2dd4de4689b0b200f97934a499 P b45a1e0cc11d4f9c9b52563a245e291b reported cstate change: term changed from 0 to 1, leader changed from <none> to b45a1e0cc11d4f9c9b52563a245e291b (127.16.178.1). New cstate: current_term: 1 leader_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45a1e0cc11d4f9c9b52563a245e291b" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 32881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 } health_report { overall_health: UNKNOWN } } }
I20260806 21:12:09.588606 29572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:09.584242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57218 (local address 127.16.178.1:32881)
0806 21:12:09.584566 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:09.584570 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:09.584584 (+ 14us) server_negotiation.cc:407] Connection header received
0806 21:12:09.584630 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:09.584632 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:09.584679 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:09.584741 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:12:09.585081 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:09.585865 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:09.587279 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:09.587654 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:09.587774 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:09.588097 (+ 323us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:12:09.588201 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:12:09.588279 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:09.588374 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 21:12:09.588446 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236}
I20260806 21:12:09.598276 29464 raft_consensus.cc:1261] T 93054c2dd4de4689b0b200f97934a499 P 4b482e8b109143a28f854c2bee2d815f [term 1 FOLLOWER]: Refusing update from remote peer b45a1e0cc11d4f9c9b52563a245e291b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:12:09.598290 29246 raft_consensus.cc:1261] T 93054c2dd4de4689b0b200f97934a499 P 18cbdda3a4834b2bbae22c4a599e7d3a [term 1 FOLLOWER]: Refusing update from remote peer b45a1e0cc11d4f9c9b52563a245e291b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:12:09.598752 29597 consensus_queue.cc:1035] T 93054c2dd4de4689b0b200f97934a499 P b45a1e0cc11d4f9c9b52563a245e291b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b482e8b109143a28f854c2bee2d815f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 44397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:12:09.599299 29597 consensus_queue.cc:1035] T 93054c2dd4de4689b0b200f97934a499 P b45a1e0cc11d4f9c9b52563a245e291b [LEADER]: Connected to new peer: Peer: permanent_uuid: "18cbdda3a4834b2bbae22c4a599e7d3a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 43593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:12:09.635820 28762 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39290:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:12:09.636010 28762 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39290:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:12:09.638211 28762 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9f793d6ef89b4cd0a78f73c7c77df3f9: Sending DeleteTablet for 3 replicas of tablet 93054c2dd4de4689b0b200f97934a499
I20260806 21:12:09.638893 28970 tablet_service.cc:1514] Processing DeleteTablet for tablet 93054c2dd4de4689b0b200f97934a499 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:12:09 UTC) from {username='oryx', principal='oryx/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:47160
I20260806 21:12:09.638947 29437 tablet_service.cc:1514] Processing DeleteTablet for tablet 93054c2dd4de4689b0b200f97934a499 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:12:09 UTC) from {username='oryx', principal='oryx/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:38944
I20260806 21:12:09.638952 29207 tablet_service.cc:1514] Processing DeleteTablet for tablet 93054c2dd4de4689b0b200f97934a499 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:12:09 UTC) from {username='oryx', principal='oryx/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:54552
I20260806 21:12:09.639148 29618 tablet_replica.cc:330] T 93054c2dd4de4689b0b200f97934a499 P b45a1e0cc11d4f9c9b52563a245e291b: stopping tablet replica
I20260806 21:12:09.639145 29622 tablet_replica.cc:330] T 93054c2dd4de4689b0b200f97934a499 P 18cbdda3a4834b2bbae22c4a599e7d3a: stopping tablet replica
I20260806 21:12:09.639264 29618 raft_consensus.cc:2229] T 93054c2dd4de4689b0b200f97934a499 P b45a1e0cc11d4f9c9b52563a245e291b [term 1 LEADER]: Raft consensus shutting down.
I20260806 21:12:09.639264 29622 raft_consensus.cc:2229] T 93054c2dd4de4689b0b200f97934a499 P 18cbdda3a4834b2bbae22c4a599e7d3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:12:09.639267 29620 tablet_replica.cc:330] T 93054c2dd4de4689b0b200f97934a499 P 4b482e8b109143a28f854c2bee2d815f: stopping tablet replica
I20260806 21:12:09.639398 29622 raft_consensus.cc:2258] T 93054c2dd4de4689b0b200f97934a499 P 18cbdda3a4834b2bbae22c4a599e7d3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:12:09.639406 29620 raft_consensus.cc:2229] T 93054c2dd4de4689b0b200f97934a499 P 4b482e8b109143a28f854c2bee2d815f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:12:09.639494 29618 raft_consensus.cc:2258] T 93054c2dd4de4689b0b200f97934a499 P b45a1e0cc11d4f9c9b52563a245e291b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:12:09.639537 29620 raft_consensus.cc:2258] T 93054c2dd4de4689b0b200f97934a499 P 4b482e8b109143a28f854c2bee2d815f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:12:09.640233 29618 ts_tablet_manager.cc:1905] T 93054c2dd4de4689b0b200f97934a499 P b45a1e0cc11d4f9c9b52563a245e291b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:12:09.640234 29622 ts_tablet_manager.cc:1905] T 93054c2dd4de4689b0b200f97934a499 P 18cbdda3a4834b2bbae22c4a599e7d3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:12:09.640264 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 28870
I20260806 21:12:09.645483 29622 ts_tablet_manager.cc:1918] T 93054c2dd4de4689b0b200f97934a499 P 18cbdda3a4834b2bbae22c4a599e7d3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 21:12:09.645596 29622 log.cc:1196] T 93054c2dd4de4689b0b200f97934a499 P 18cbdda3a4834b2bbae22c4a599e7d3a: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-1/wal/wals/93054c2dd4de4689b0b200f97934a499
I20260806 21:12:09.645818 29622 ts_tablet_manager.cc:1939] T 93054c2dd4de4689b0b200f97934a499 P 18cbdda3a4834b2bbae22c4a599e7d3a: Deleting consensus metadata
I20260806 21:12:09.647869 29620 ts_tablet_manager.cc:1905] T 93054c2dd4de4689b0b200f97934a499 P 4b482e8b109143a28f854c2bee2d815f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:12:09.650357 29620 ts_tablet_manager.cc:1918] T 93054c2dd4de4689b0b200f97934a499 P 4b482e8b109143a28f854c2bee2d815f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 21:12:09.650467 29620 log.cc:1196] T 93054c2dd4de4689b0b200f97934a499 P 4b482e8b109143a28f854c2bee2d815f: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786050686797304-17096-0/minicluster-data/ts-2/wal/wals/93054c2dd4de4689b0b200f97934a499
I20260806 21:12:09.650693 29620 ts_tablet_manager.cc:1939] T 93054c2dd4de4689b0b200f97934a499 P 4b482e8b109143a28f854c2bee2d815f: Deleting consensus metadata
I20260806 21:12:09.651634 28741 catalog_manager.cc:4841] TS 4b482e8b109143a28f854c2bee2d815f (127.16.178.3:44397): tablet 93054c2dd4de4689b0b200f97934a499 (table test-table [id=962c786da6bc4630a830e939fdab01eb]) successfully deleted
I20260806 21:12:09.652539 28742 catalog_manager.cc:4841] TS 18cbdda3a4834b2bbae22c4a599e7d3a (127.16.178.2:43593): tablet 93054c2dd4de4689b0b200f97934a499 (table test-table [id=962c786da6bc4630a830e939fdab01eb]) successfully deleted
W20260806 21:12:09.661481 28743 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.178.1:32881 (error 108)
I20260806 21:12:09.662276 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 29093
I20260806 21:12:09.665411 29728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:09.662459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:32881 (local address 127.0.0.1:57234)
0806 21:12:09.665176 (+ 2717us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:09.665281 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.1:32881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2604,"thread_start_us":60,"threads_started":1}
W20260806 21:12:09.667611 28743 catalog_manager.cc:4584] TS b45a1e0cc11d4f9c9b52563a245e291b (127.16.178.1:32881): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 93054c2dd4de4689b0b200f97934a499: Network error: Client connection negotiation failed: client connection to 127.16.178.1:32881: connect: Connection refused (error 111)
I20260806 21:12:09.675452 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 29336
I20260806 21:12:09.696858 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 28712
2026-08-06T21:12:09Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3346 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786050686797304-17096-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:12:09 dist-test-slave-lgr5 krb5kdc[29733](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:12:09 dist-test-slave-lgr5 krb5kdc[29733](info): set up 2 sockets
Aug 06 21:12:09 dist-test-slave-lgr5 krb5kdc[29733](info): commencing operation
krb5kdc: starting...
W20260806 21:12:10.878460 17096 mini_kdc.cc:121] Time spent starting KDC: real 1.144s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Aug 06 21:12:10 dist-test-slave-lgr5 krb5kdc[29733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050730, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:12:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:12:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.178.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
I20260806 21:12:11.015110 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:43725
--webserver_interface=127.16.178.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:36119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:43725
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:12:11.129998 29753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:12:11.130336 29753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:12:11.130395 29753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:12:11.134629 29753 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:12:11.134723 29753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:12:11.134753 29753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:12:11.134774 29753 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:12:11.134796 29753 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:12:11.140599 29753 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:36119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:43725
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:43725
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.178.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:12:11.141767 29753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:12:11.142835 29753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:12:11.151458 29765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:11.151542 29760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:11.151510 29762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:11.151680 29753 server_base.cc:1034] running on GCE node
I20260806 21:12:11.152266 29753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:12:11.153419 29753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:12:11.154614 29753 hybrid_clock.cc:648] HybridClock initialized: now 1786050731154599 us; error 28 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-06T21:12:11Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1451 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:12:11 dist-test-slave-lgr5 krb5kdc[29773](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:12:11 dist-test-slave-lgr5 krb5kdc[29773](info): set up 2 sockets
Aug 06 21:12:11 dist-test-slave-lgr5 krb5kdc[29773](info): commencing operation
krb5kdc: starting...
W20260806 21:12:12.329329 17096 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Aug 06 21:12:12 dist-test-slave-lgr5 krb5kdc[29773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050732, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:12:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:12:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.178.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
I20260806 21:12:12.467592 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:44553
--webserver_interface=127.16.178.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:32925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:44553
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:12:12.589618 29794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:12:12.589922 29794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:12:12.589972 29794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:12:12.594173 29794 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:12:12.594266 29794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:12:12.594295 29794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:12:12.594317 29794 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:12:12.594338 29794 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:12:12.599982 29794 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:32925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:44553
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:44553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.178.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:12:12.601168 29794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:12:12.602209 29794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:12:12.610684 29800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:12.610776 29801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:12.610728 29805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:12.610836 29794 server_base.cc:1034] running on GCE node
I20260806 21:12:12.611447 29794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:12:12.612612 29794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:12:12.613785 29794 hybrid_clock.cc:648] HybridClock initialized: now 1786050732613766 us; error 29 us; skew 500 ppm
Aug 06 21:12:12 dist-test-slave-lgr5 krb5kdc[29773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050732, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:12:12.616957 29794 init.cc:377] Logged in from keytab as kudu/127.16.178.62@KRBTEST.COM (short username kudu)
I20260806 21:12:12.618103 29794 webserver.cc:460] Webserver started at http://127.16.178.62:38059/ using document root <none> and password file <none>
I20260806 21:12:12.618702 29794 fs_manager.cc:359] Metadata directory not provided
I20260806 21:12:12.618777 29794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:12:12.619009 29794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:12:12.621315 29794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "c24b81726c01467ca00351c4c8ceb9bd"
format_stamp: "Formatted at 2026-08-06 21:12:12 on dist-test-slave-lgr5"
server_key: "f82b4b9eec91baec575166603cae1ad9"
server_key_iv: "e8023e5cba8d03e0334af06962d23e15"
server_key_version: "encryptionkey@0"
I20260806 21:12:12.621904 29794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "c24b81726c01467ca00351c4c8ceb9bd"
format_stamp: "Formatted at 2026-08-06 21:12:12 on dist-test-slave-lgr5"
server_key: "f82b4b9eec91baec575166603cae1ad9"
server_key_iv: "e8023e5cba8d03e0334af06962d23e15"
server_key_version: "encryptionkey@0"
I20260806 21:12:12.625962 29794 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:12:12.628509 29820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:12.629806 29794 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 21:12:12.629923 29794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "c24b81726c01467ca00351c4c8ceb9bd"
format_stamp: "Formatted at 2026-08-06 21:12:12 on dist-test-slave-lgr5"
server_key: "f82b4b9eec91baec575166603cae1ad9"
server_key_iv: "e8023e5cba8d03e0334af06962d23e15"
server_key_version: "encryptionkey@0"
I20260806 21:12:12.630046 29794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:12:12.650468 29794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:12:12.653751 29794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:12:12.653940 29794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:12:12.661429 29794 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.62:44553
I20260806 21:12:12.661477 29897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.62:44553 every 8 connection(s)
I20260806 21:12:12.662465 29794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:12:12.665577 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 29794
I20260806 21:12:12.665625 29899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:12:12.665705 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:12:12.666010 17096 external_mini_cluster.cc:1428] Setting key d20161b4c6bb90c67d7b4c4a168430f3
I20260806 21:12:12.673772 29899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c24b81726c01467ca00351c4c8ceb9bd: Bootstrap starting.
Aug 06 21:12:12 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050732, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:12:12.676929 29899 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c24b81726c01467ca00351c4c8ceb9bd: Neither blocks nor log segments found. Creating new log.
I20260806 21:12:12.677953 29899 log.cc:824] T 00000000000000000000000000000000 P c24b81726c01467ca00351c4c8ceb9bd: Log is configured to *not* fsync() on all Append() calls
I20260806 21:12:12.680801 29899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c24b81726c01467ca00351c4c8ceb9bd: No bootstrap required, opened a new log
I20260806 21:12:12.683218 29903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:12.667539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33086 (local address 127.16.178.62:44553)
0806 21:12:12.668053 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:12.668074 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:12.668160 (+ 86us) server_negotiation.cc:407] Connection header received
0806 21:12:12.669257 (+ 1097us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:12.669285 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:12.669749 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:12.670175 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:12.670923 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:12.672532 (+ 1609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:12.673554 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:12.674277 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:12.674468 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:12.677386 (+ 2918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:12.677394 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:12.677414 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:12.677457 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:12.679551 (+ 2094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:12.680119 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:12.680126 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:12.680134 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:12.680303 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:12.680625 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:12.680628 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:12.680630 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:12.681255 (+ 625us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:12.681467 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:12.681810 (+ 343us) server_negotiation.cc:299] Negotiation successful
0806 21:12:12.682161 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":156,"threads_started":1}
I20260806 21:12:12.685734 29899 raft_consensus.cc:348] T 00000000000000000000000000000000 P c24b81726c01467ca00351c4c8ceb9bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c24b81726c01467ca00351c4c8ceb9bd" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44553 } }
I20260806 21:12:12.686048 29899 raft_consensus.cc:374] T 00000000000000000000000000000000 P c24b81726c01467ca00351c4c8ceb9bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:12:12.686094 29899 raft_consensus.cc:729] T 00000000000000000000000000000000 P c24b81726c01467ca00351c4c8ceb9bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c24b81726c01467ca00351c4c8ceb9bd, State: Initialized, Role: FOLLOWER
I20260806 21:12:12.686519 29899 consensus_queue.cc:260] T 00000000000000000000000000000000 P c24b81726c01467ca00351c4c8ceb9bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c24b81726c01467ca00351c4c8ceb9bd" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44553 } }
I20260806 21:12:12.686661 29899 raft_consensus.cc:388] T 00000000000000000000000000000000 P c24b81726c01467ca00351c4c8ceb9bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:12:12.686714 29899 raft_consensus.cc:482] T 00000000000000000000000000000000 P c24b81726c01467ca00351c4c8ceb9bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:12:12.686797 29899 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c24b81726c01467ca00351c4c8ceb9bd [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:12:12.687875 29899 raft_consensus.cc:504] T 00000000000000000000000000000000 P c24b81726c01467ca00351c4c8ceb9bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c24b81726c01467ca00351c4c8ceb9bd" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44553 } }
I20260806 21:12:12.688265 29899 leader_election.cc:302] T 00000000000000000000000000000000 P c24b81726c01467ca00351c4c8ceb9bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c24b81726c01467ca00351c4c8ceb9bd; no voters:
I20260806 21:12:12.688624 29899 leader_election.cc:288] T 00000000000000000000000000000000 P c24b81726c01467ca00351c4c8ceb9bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:12:12.688747 29906 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c24b81726c01467ca00351c4c8ceb9bd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:12:12.688987 29906 raft_consensus.cc:686] T 00000000000000000000000000000000 P c24b81726c01467ca00351c4c8ceb9bd [term 1 LEADER]: Becoming Leader. State: Replica: c24b81726c01467ca00351c4c8ceb9bd, State: Running, Role: LEADER
I20260806 21:12:12.689369 29906 consensus_queue.cc:237] T 00000000000000000000000000000000 P c24b81726c01467ca00351c4c8ceb9bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c24b81726c01467ca00351c4c8ceb9bd" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44553 } }
I20260806 21:12:12.689905 29899 sys_catalog.cc:564] T 00000000000000000000000000000000 P c24b81726c01467ca00351c4c8ceb9bd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:12:12.691021 29909 sys_catalog.cc:455] T 00000000000000000000000000000000 P c24b81726c01467ca00351c4c8ceb9bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader c24b81726c01467ca00351c4c8ceb9bd. Latest consensus state: current_term: 1 leader_uuid: "c24b81726c01467ca00351c4c8ceb9bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c24b81726c01467ca00351c4c8ceb9bd" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44553 } } }
I20260806 21:12:12.691169 29909 sys_catalog.cc:458] T 00000000000000000000000000000000 P c24b81726c01467ca00351c4c8ceb9bd [sys.catalog]: This master's current role is: LEADER
I20260806 21:12:12.691534 29907 sys_catalog.cc:455] T 00000000000000000000000000000000 P c24b81726c01467ca00351c4c8ceb9bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c24b81726c01467ca00351c4c8ceb9bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c24b81726c01467ca00351c4c8ceb9bd" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44553 } } }
I20260806 21:12:12.691694 29907 sys_catalog.cc:458] T 00000000000000000000000000000000 P c24b81726c01467ca00351c4c8ceb9bd [sys.catalog]: This master's current role is: LEADER
I20260806 21:12:12.691601 29918 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:12:12.694864 29918 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:12:12.700815 29918 catalog_manager.cc:1347] Generated new cluster ID: a3d8e0c9b7844febb9c9c82a387d78de
I20260806 21:12:12.700911 29918 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:12:12.710820 29918 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:12:12.711776 29918 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:12:12.719859 29918 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c24b81726c01467ca00351c4c8ceb9bd: Generated new TSK 0
I20260806 21:12:12.720646 29918 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.178.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab.
Entry for principal kudu/127.16.178.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab.
I20260806 21:12:12.759455 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.1:0
--local_ip_for_outbound_sockets=127.16.178.1
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:44553
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab
--principal=kudu/127.16.178.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:32925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:12:12.879474 29933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:12:12.879748 29933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:12:12.879793 29933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:12:12.883831 29933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:12:12.883916 29933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:12:12.883984 29933 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.1
I20260806 21:12:12.889278 29933 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:32925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab
--principal=kudu/127.16.178.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:44553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.1
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:12:12.890359 29933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:12:12.891428 29933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:12:12.900537 29943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:12.900632 29933 server_base.cc:1034] running on GCE node
W20260806 21:12:12.900537 29947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:12.900664 29942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:12.901156 29933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:12:12.901757 29933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:12:12.902918 29933 hybrid_clock.cc:648] HybridClock initialized: now 1786050732902897 us; error 31 us; skew 500 ppm
Aug 06 21:12:12 dist-test-slave-lgr5 krb5kdc[29773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050732, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:12:12.905997 29933 init.cc:377] Logged in from keytab as kudu/127.16.178.1@KRBTEST.COM (short username kudu)
I20260806 21:12:12.907095 29933 webserver.cc:460] Webserver started at http://127.16.178.1:34879/ using document root <none> and password file <none>
I20260806 21:12:12.907652 29933 fs_manager.cc:359] Metadata directory not provided
I20260806 21:12:12.907739 29933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:12:12.908049 29933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:12:12.910092 29933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-0/data/instance:
uuid: "35d07d8e5bb048598b211b5d50ff137c"
format_stamp: "Formatted at 2026-08-06 21:12:12 on dist-test-slave-lgr5"
server_key: "0ef5132c0743a323e2e815e13b3ed350"
server_key_iv: "6150d147a37c7c25e2aed47d639be9bf"
server_key_version: "encryptionkey@0"
I20260806 21:12:12.910601 29933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance:
uuid: "35d07d8e5bb048598b211b5d50ff137c"
format_stamp: "Formatted at 2026-08-06 21:12:12 on dist-test-slave-lgr5"
server_key: "0ef5132c0743a323e2e815e13b3ed350"
server_key_iv: "6150d147a37c7c25e2aed47d639be9bf"
server_key_version: "encryptionkey@0"
I20260806 21:12:12.914152 29933 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 21:12:12.916554 29957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:12.917913 29933 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 21:12:12.918011 29933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-0/wal
uuid: "35d07d8e5bb048598b211b5d50ff137c"
format_stamp: "Formatted at 2026-08-06 21:12:12 on dist-test-slave-lgr5"
server_key: "0ef5132c0743a323e2e815e13b3ed350"
server_key_iv: "6150d147a37c7c25e2aed47d639be9bf"
server_key_version: "encryptionkey@0"
I20260806 21:12:12.918097 29933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:12:12.935109 29933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:12:12.938395 29933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:12:12.938570 29933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:12:12.939162 29933 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:12:12.940202 29933 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:12:12.940268 29933 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:12.940316 29933 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:12:12.940344 29933 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:12.949931 29933 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.1:46769
I20260806 21:12:12.949955 30142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.1:46769 every 8 connection(s)
I20260806 21:12:12.951067 29933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
I20260806 21:12:12.957470 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 29933
I20260806 21:12:12.957597 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance
I20260806 21:12:12.957881 17096 external_mini_cluster.cc:1428] Setting key 24df39062d698909c8c23fcb1114f97a
Aug 06 21:12:12 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050732, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:12:12.965615 29903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:12.953286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:60899 (local address 127.16.178.62:44553)
0806 21:12:12.953478 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:12.953483 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:12.954667 (+ 1184us) server_negotiation.cc:407] Connection header received
0806 21:12:12.955580 (+ 913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:12.955583 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:12.955634 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:12.955709 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:12.956991 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:12.957560 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:12.958982 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:12.959263 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:12.959362 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:12.961919 (+ 2557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:12.961923 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:12.961926 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:12.961957 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:12.963407 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:12.963934 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:12.963938 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:12.963940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:12.964033 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:12.964390 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:12.964393 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:12.964395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:12.964642 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:12.964747 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:12.965275 (+ 528us) server_negotiation.cc:299] Negotiation successful
0806 21:12:12.965430 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 21:12:12.966387 30148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:12.953716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.1:60899)
0806 21:12:12.954492 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:12.954543 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:12.955395 (+ 852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:12.955882 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:12.955889 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:12.956324 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:12.956858 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:12.956874 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:12.957723 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:12.957726 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:12.958825 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:12.958838 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:12.959434 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:12.959439 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:12.959622 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:12.960221 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:12.960251 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:12.961722 (+ 1471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:12:12.963547 (+ 1825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:12.963553 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:12.963569 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:12.963830 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:12.964145 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:12.964147 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:12.964148 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:12.964289 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:12.964746 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:12:12.964752 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:12:12.965024 (+ 272us) client_negotiation.cc:769] Sending connection context
0806 21:12:12.965193 (+ 169us) client_negotiation.cc:240] Negotiation successful
0806 21:12:12.965472 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":143,"threads_started":1}
I20260806 21:12:12.967707 30143 heartbeater.cc:347] Connected to a master server at 127.16.178.62:44553
I20260806 21:12:12.967947 30143 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.16.178.2@KRBTEST.COM; defaulting to no policy
I20260806 21:12:12.968505 30143 heartbeater.cc:511] Master 127.16.178.62:44553 requested a full tablet report, sending...
I20260806 21:12:12.970152 29852 ts_manager.cc:194] Registered new tserver with Master: 35d07d8e5bb048598b211b5d50ff137c (127.16.178.1:46769)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.2@KRBTEST.COM" created.
I20260806 21:12:12.971379 29852 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.1@KRBTEST.COM'} at 127.16.178.1:60899
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab.
Entry for principal kudu/127.16.178.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab.
I20260806 21:12:12.987363 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.2:0
--local_ip_for_outbound_sockets=127.16.178.2
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:44553
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab
--principal=kudu/127.16.178.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:32925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:12:13.107614 30152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:12:13.107932 30152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:12:13.107982 30152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:12:13.112062 30152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:12:13.112160 30152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:12:13.112243 30152 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.2
I20260806 21:12:13.117782 30152 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:32925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab
--principal=kudu/127.16.178.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:44553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30152
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.2
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:12:13.119031 30152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:12:13.120258 30152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:12:13.131433 30160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:13.131433 30158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:13.131436 30163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:13.131575 30152 server_base.cc:1034] running on GCE node
I20260806 21:12:13.132246 30152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:12:13.133028 30152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:12:13.134228 30152 hybrid_clock.cc:648] HybridClock initialized: now 1786050733134199 us; error 37 us; skew 500 ppm
Aug 06 21:12:13 dist-test-slave-lgr5 krb5kdc[29773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050733, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:12:13.137323 30152 init.cc:377] Logged in from keytab as kudu/127.16.178.2@KRBTEST.COM (short username kudu)
I20260806 21:12:13.138309 30152 webserver.cc:460] Webserver started at http://127.16.178.2:33599/ using document root <none> and password file <none>
I20260806 21:12:13.138983 30152 fs_manager.cc:359] Metadata directory not provided
I20260806 21:12:13.139062 30152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:12:13.139256 30152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:12:13.141659 30152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-1/data/instance:
uuid: "0aa0fcaf7fb0482a85970a939ea2ac2c"
format_stamp: "Formatted at 2026-08-06 21:12:13 on dist-test-slave-lgr5"
server_key: "bd300968564dac0d2e12c4c84fa74114"
server_key_iv: "4fa5167ac9a51bf9ddedba2d543aa1c5"
server_key_version: "encryptionkey@0"
I20260806 21:12:13.142266 30152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance:
uuid: "0aa0fcaf7fb0482a85970a939ea2ac2c"
format_stamp: "Formatted at 2026-08-06 21:12:13 on dist-test-slave-lgr5"
server_key: "bd300968564dac0d2e12c4c84fa74114"
server_key_iv: "4fa5167ac9a51bf9ddedba2d543aa1c5"
server_key_version: "encryptionkey@0"
I20260806 21:12:13.146471 30152 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:12:13.149019 30175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:13.150231 30152 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 21:12:13.150358 30152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-1/wal
uuid: "0aa0fcaf7fb0482a85970a939ea2ac2c"
format_stamp: "Formatted at 2026-08-06 21:12:13 on dist-test-slave-lgr5"
server_key: "bd300968564dac0d2e12c4c84fa74114"
server_key_iv: "4fa5167ac9a51bf9ddedba2d543aa1c5"
server_key_version: "encryptionkey@0"
I20260806 21:12:13.150453 30152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:12:13.173408 30152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:12:13.177110 30152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:12:13.177314 30152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:12:13.178045 30152 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:12:13.179165 30152 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:12:13.179235 30152 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:13.179301 30152 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:12:13.179329 30152 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:13.190624 30152 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.2:44659
I20260806 21:12:13.190685 30352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.2:44659 every 8 connection(s)
I20260806 21:12:13.191707 30152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
I20260806 21:12:13.195583 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 30152
I20260806 21:12:13.195729 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance
I20260806 21:12:13.196044 17096 external_mini_cluster.cc:1428] Setting key 971a23427c6786270438eee2658d6b3e
Aug 06 21:12:13 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050733, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.2@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.178.3@KRBTEST.COM; defaulting to no policy
I20260806 21:12:13.210378 29903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:13.193914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:43197 (local address 127.16.178.62:44553)
0806 21:12:13.194059 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:13.194064 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:13.195535 (+ 1471us) server_negotiation.cc:407] Connection header received
0806 21:12:13.196943 (+ 1408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:13.196947 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:13.197002 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:13.197091 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:13.199090 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:13.199839 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:13.201248 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:13.201498 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:13.201633 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:13.205282 (+ 3649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:13.205291 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:13.205293 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:13.205330 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:13.207027 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:13.207947 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:13.207952 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:13.207954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:13.208164 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:13.208742 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:13.208746 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:13.208748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:13.209009 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:13.209129 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:13.209934 (+ 805us) server_negotiation.cc:299] Negotiation successful
0806 21:12:13.210116 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.3@KRBTEST.COM" created.
I20260806 21:12:13.211493 30356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:13.194415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.2:43197)
0806 21:12:13.195324 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:13.195406 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:13.196636 (+ 1230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:13.197350 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:13.197375 (+ 25us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:13.197964 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:13.198872 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:13.198899 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:13.200031 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:13.200035 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:13.201101 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:13.201115 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:13.201706 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:13.201711 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:13.201973 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:13.202732 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:13.202757 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:13.204964 (+ 2207us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:12:13.207214 (+ 2250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:13.207225 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:13.207250 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:13.207741 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:13.208300 (+ 559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:13.208305 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:13.208307 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:13.208564 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:13.209131 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:12:13.209141 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:12:13.209592 (+ 451us) client_negotiation.cc:769] Sending connection context
0806 21:12:13.209866 (+ 274us) client_negotiation.cc:240] Negotiation successful
0806 21:12:13.210333 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"thread_start_us":140,"threads_started":1}
I20260806 21:12:13.212845 30353 heartbeater.cc:347] Connected to a master server at 127.16.178.62:44553
I20260806 21:12:13.213127 30353 heartbeater.cc:464] Registering TS with master...
I20260806 21:12:13.213708 30353 heartbeater.cc:511] Master 127.16.178.62:44553 requested a full tablet report, sending...
I20260806 21:12:13.214900 29852 ts_manager.cc:194] Registered new tserver with Master: 0aa0fcaf7fb0482a85970a939ea2ac2c (127.16.178.2:44659)
I20260806 21:12:13.215529 29852 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.2@KRBTEST.COM'} at 127.16.178.2:43197
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab.
Entry for principal kudu/127.16.178.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab.
I20260806 21:12:13.231215 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.3:0
--local_ip_for_outbound_sockets=127.16.178.3
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:44553
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab
--principal=kudu/127.16.178.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:32925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:12:13.351905 30361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:12:13.352267 30361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:12:13.352335 30361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:12:13.356635 30361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:12:13.356715 30361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:12:13.356802 30361 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.3
I20260806 21:12:13.362361 30361 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:32925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab
--principal=kudu/127.16.178.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:44553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30361
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.3
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:12:13.363577 30361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:12:13.364689 30361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:12:13.374872 30373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:13.374919 30371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:13.374949 30370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:13.375007 30361 server_base.cc:1034] running on GCE node
I20260806 21:12:13.375713 30361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:12:13.376426 30361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:12:13.377645 30361 hybrid_clock.cc:648] HybridClock initialized: now 1786050733377624 us; error 32 us; skew 500 ppm
Aug 06 21:12:13 dist-test-slave-lgr5 krb5kdc[29773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050733, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:12:13.380548 30361 init.cc:377] Logged in from keytab as kudu/127.16.178.3@KRBTEST.COM (short username kudu)
I20260806 21:12:13.381469 30361 webserver.cc:460] Webserver started at http://127.16.178.3:35367/ using document root <none> and password file <none>
I20260806 21:12:13.382047 30361 fs_manager.cc:359] Metadata directory not provided
I20260806 21:12:13.382118 30361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:12:13.382325 30361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:12:13.384517 30361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-2/data/instance:
uuid: "ac8be99728bb4ae585127dc7665a56c7"
format_stamp: "Formatted at 2026-08-06 21:12:13 on dist-test-slave-lgr5"
server_key: "77f245680414f94fc3ba742896ecdaab"
server_key_iv: "e9bd65c94bb0437c736ac4a6a00f3114"
server_key_version: "encryptionkey@0"
I20260806 21:12:13.385082 30361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance:
uuid: "ac8be99728bb4ae585127dc7665a56c7"
format_stamp: "Formatted at 2026-08-06 21:12:13 on dist-test-slave-lgr5"
server_key: "77f245680414f94fc3ba742896ecdaab"
server_key_iv: "e9bd65c94bb0437c736ac4a6a00f3114"
server_key_version: "encryptionkey@0"
I20260806 21:12:13.389251 30361 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 21:12:13.392000 30384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:13.393376 30361 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 21:12:13.393501 30361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-2/wal
uuid: "ac8be99728bb4ae585127dc7665a56c7"
format_stamp: "Formatted at 2026-08-06 21:12:13 on dist-test-slave-lgr5"
server_key: "77f245680414f94fc3ba742896ecdaab"
server_key_iv: "e9bd65c94bb0437c736ac4a6a00f3114"
server_key_version: "encryptionkey@0"
I20260806 21:12:13.393620 30361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:12:13.417940 30361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:12:13.421654 30361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:12:13.421851 30361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:12:13.422529 30361 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:12:13.423496 30361 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:12:13.423561 30361 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:13.423620 30361 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:12:13.423650 30361 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:13.432801 30361 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.3:34007
I20260806 21:12:13.432826 30553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.3:34007 every 8 connection(s)
I20260806 21:12:13.433840 30361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
I20260806 21:12:13.439688 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 30361
I20260806 21:12:13.439862 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance
I20260806 21:12:13.440255 17096 external_mini_cluster.cc:1428] Setting key 5dd86f422e3ed365e9905e02bcc6f081
Aug 06 21:12:13 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050733, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.3@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:12:13.448997 29903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:13.435792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:58895 (local address 127.16.178.62:44553)
0806 21:12:13.435935 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:13.435939 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:13.437294 (+ 1355us) server_negotiation.cc:407] Connection header received
0806 21:12:13.438244 (+ 950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:13.438248 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:13.438357 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:13.438444 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:13.439948 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:13.440548 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:13.442104 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:13.442341 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:13.442454 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:13.445267 (+ 2813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:13.445271 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:13.445273 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:13.445303 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:13.446697 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:13.447290 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:13.447293 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:13.447294 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:13.447383 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:13.447748 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:13.447752 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:13.447754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:13.447969 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:13.448060 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:13.448670 (+ 610us) server_negotiation.cc:299] Negotiation successful
0806 21:12:13.448805 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 21:12:13.449923 30558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:13.436157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.3:58895)
0806 21:12:13.437101 (+ 944us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:13.437148 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:13.438022 (+ 874us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:13.438641 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:13.438649 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:13.439054 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:13.439739 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:13.439765 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:13.440711 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:13.440715 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:13.441973 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:13.441987 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:13.442535 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:13.442540 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:13.442732 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:13.443370 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:13.443389 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:13.445036 (+ 1647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:12:13.446847 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:13.446854 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:13.446873 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:13.447180 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:13.447486 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:13.447490 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:13.447492 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:13.447651 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:13.448098 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:12:13.448106 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:12:13.448398 (+ 292us) client_negotiation.cc:769] Sending connection context
0806 21:12:13.448607 (+ 209us) client_negotiation.cc:240] Negotiation successful
0806 21:12:13.448995 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":731,"thread_start_us":533,"threads_started":1}
I20260806 21:12:13.451264 30556 heartbeater.cc:347] Connected to a master server at 127.16.178.62:44553
I20260806 21:12:13.451551 30556 heartbeater.cc:464] Registering TS with master...
I20260806 21:12:13.452196 30556 heartbeater.cc:511] Master 127.16.178.62:44553 requested a full tablet report, sending...
I20260806 21:12:13.453558 29852 ts_manager.cc:194] Registered new tserver with Master: ac8be99728bb4ae585127dc7665a56c7 (127.16.178.3:34007)
I20260806 21:12:13.454216 29852 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.3@KRBTEST.COM'} at 127.16.178.3:58895
I20260806 21:12:13.454809 17096 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.16.178.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.178.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.178.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.1.keytab.
Entry for principal oryx/127.16.178.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.1.keytab.
WARNING: no policy specified for oryx/127.16.178.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.178.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.178.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.2.keytab.
Entry for principal oryx/127.16.178.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.2.keytab.
WARNING: no policy specified for oryx/127.16.178.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.178.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.178.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.3.keytab.
Entry for principal oryx/127.16.178.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.3.keytab.
I20260806 21:12:13.539695 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 29933
I20260806 21:12:13.555861 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 30152
I20260806 21:12:13.570025 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 30361
I20260806 21:12:13.594004 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 29794
I20260806 21:12:13.606046 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:44553
--webserver_interface=127.16.178.62
--webserver_port=38059
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:32925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:44553
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:12:13.719853 30570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:12:13.720218 30570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:12:13.720263 30570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:12:13.724184 30570 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:12:13.724238 30570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:12:13.724257 30570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:12:13.724272 30570 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:12:13.724285 30570 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:12:13.729681 30570 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:32925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:44553
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:44553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.178.62
--webserver_port=38059
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30570
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:12:13.730870 30570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:12:13.731892 30570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:12:13.740429 30575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:13.740509 30577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:13.740495 30570 server_base.cc:1034] running on GCE node
W20260806 21:12:13.740538 30580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:13.741199 30570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:12:13.742298 30570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:12:13.743487 30570 hybrid_clock.cc:648] HybridClock initialized: now 1786050733743459 us; error 41 us; skew 500 ppm
Aug 06 21:12:13 dist-test-slave-lgr5 krb5kdc[29773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050733, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:12:13.746368 30570 init.cc:377] Logged in from keytab as kudu/127.16.178.62@KRBTEST.COM (short username kudu)
I20260806 21:12:13.747363 30570 webserver.cc:460] Webserver started at http://127.16.178.62:38059/ using document root <none> and password file <none>
I20260806 21:12:13.747965 30570 fs_manager.cc:359] Metadata directory not provided
I20260806 21:12:13.748056 30570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:12:13.752146 30570 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260806 21:12:13.754406 30590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:13.755748 30570 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260806 21:12:13.755878 30570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "c24b81726c01467ca00351c4c8ceb9bd"
format_stamp: "Formatted at 2026-08-06 21:12:12 on dist-test-slave-lgr5"
server_key: "f82b4b9eec91baec575166603cae1ad9"
server_key_iv: "e8023e5cba8d03e0334af06962d23e15"
server_key_version: "encryptionkey@0"
I20260806 21:12:13.756619 30570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:12:13.772995 30570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:12:13.776412 30570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:12:13.776628 30570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:12:13.784039 30570 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.62:44553
I20260806 21:12:13.784049 30672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.62:44553 every 8 connection(s)
I20260806 21:12:13.785209 30570 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:12:13.789752 30678 sys_catalog.cc:263] Verifying existing consensus state
I20260806 21:12:13.791535 30678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c24b81726c01467ca00351c4c8ceb9bd: Bootstrap starting.
I20260806 21:12:13.793186 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 30570
I20260806 21:12:13.793735 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.1:46769
--local_ip_for_outbound_sockets=127.16.178.1
--tserver_master_addrs=127.16.178.62:44553
--webserver_port=34879
--webserver_interface=127.16.178.1
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab
--principal=kudu/127.16.178.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:32925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.16.178.1
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 21:12:13.810992 30678 log.cc:824] T 00000000000000000000000000000000 P c24b81726c01467ca00351c4c8ceb9bd: Log is configured to *not* fsync() on all Append() calls
I20260806 21:12:13.818466 30678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c24b81726c01467ca00351c4c8ceb9bd: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 21:12:13.818887 30678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c24b81726c01467ca00351c4c8ceb9bd: Bootstrap complete.
I20260806 21:12:13.823352 30678 raft_consensus.cc:348] T 00000000000000000000000000000000 P c24b81726c01467ca00351c4c8ceb9bd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c24b81726c01467ca00351c4c8ceb9bd" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44553 } }
I20260806 21:12:13.823824 30678 raft_consensus.cc:729] T 00000000000000000000000000000000 P c24b81726c01467ca00351c4c8ceb9bd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c24b81726c01467ca00351c4c8ceb9bd, State: Initialized, Role: FOLLOWER
I20260806 21:12:13.824419 30678 consensus_queue.cc:260] T 00000000000000000000000000000000 P c24b81726c01467ca00351c4c8ceb9bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c24b81726c01467ca00351c4c8ceb9bd" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44553 } }
I20260806 21:12:13.824553 30678 raft_consensus.cc:388] T 00000000000000000000000000000000 P c24b81726c01467ca00351c4c8ceb9bd [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:12:13.824622 30678 raft_consensus.cc:482] T 00000000000000000000000000000000 P c24b81726c01467ca00351c4c8ceb9bd [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:12:13.824718 30678 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c24b81726c01467ca00351c4c8ceb9bd [term 1 FOLLOWER]: Advancing to term 2
I20260806 21:12:13.826782 30678 raft_consensus.cc:504] T 00000000000000000000000000000000 P c24b81726c01467ca00351c4c8ceb9bd [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c24b81726c01467ca00351c4c8ceb9bd" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44553 } }
I20260806 21:12:13.827137 30678 leader_election.cc:302] T 00000000000000000000000000000000 P c24b81726c01467ca00351c4c8ceb9bd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c24b81726c01467ca00351c4c8ceb9bd; no voters:
I20260806 21:12:13.827452 30678 leader_election.cc:288] T 00000000000000000000000000000000 P c24b81726c01467ca00351c4c8ceb9bd [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 21:12:13.827567 30684 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c24b81726c01467ca00351c4c8ceb9bd [term 2 FOLLOWER]: Leader election won for term 2
I20260806 21:12:13.827778 30684 raft_consensus.cc:686] T 00000000000000000000000000000000 P c24b81726c01467ca00351c4c8ceb9bd [term 2 LEADER]: Becoming Leader. State: Replica: c24b81726c01467ca00351c4c8ceb9bd, State: Running, Role: LEADER
I20260806 21:12:13.828130 30684 consensus_queue.cc:237] T 00000000000000000000000000000000 P c24b81726c01467ca00351c4c8ceb9bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c24b81726c01467ca00351c4c8ceb9bd" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44553 } }
I20260806 21:12:13.828907 30678 sys_catalog.cc:564] T 00000000000000000000000000000000 P c24b81726c01467ca00351c4c8ceb9bd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:12:13.829744 30685 sys_catalog.cc:455] T 00000000000000000000000000000000 P c24b81726c01467ca00351c4c8ceb9bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c24b81726c01467ca00351c4c8ceb9bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c24b81726c01467ca00351c4c8ceb9bd" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44553 } } }
I20260806 21:12:13.829901 30685 sys_catalog.cc:458] T 00000000000000000000000000000000 P c24b81726c01467ca00351c4c8ceb9bd [sys.catalog]: This master's current role is: LEADER
I20260806 21:12:13.830219 30686 sys_catalog.cc:455] T 00000000000000000000000000000000 P c24b81726c01467ca00351c4c8ceb9bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader c24b81726c01467ca00351c4c8ceb9bd. Latest consensus state: current_term: 2 leader_uuid: "c24b81726c01467ca00351c4c8ceb9bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c24b81726c01467ca00351c4c8ceb9bd" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 44553 } } }
I20260806 21:12:13.830317 30686 sys_catalog.cc:458] T 00000000000000000000000000000000 P c24b81726c01467ca00351c4c8ceb9bd [sys.catalog]: This master's current role is: LEADER
I20260806 21:12:13.830994 30699 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:12:13.834179 30699 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:12:13.834816 30699 catalog_manager.cc:1259] Loaded cluster ID: a3d8e0c9b7844febb9c9c82a387d78de
I20260806 21:12:13.834879 30699 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:12:13.836746 30699 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:12:13.837339 30699 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c24b81726c01467ca00351c4c8ceb9bd: Loaded TSK: 0
I20260806 21:12:13.838336 30699 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 21:12:13.927317 30681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:12:13.927623 30681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:12:13.927686 30681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:12:13.932060 30681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:12:13.932168 30681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:12:13.932251 30681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.1
I20260806 21:12:13.937717 30681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:32925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.1.keytab
--principal=oryx/127.16.178.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.1:46769
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.178.1
--webserver_port=34879
--tserver_master_addrs=127.16.178.62:44553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.1
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:12:13.938817 30681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:12:13.939867 30681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:12:13.950201 30727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:13.950127 30723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:13.950143 30724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:13.950565 30681 server_base.cc:1034] running on GCE node
I20260806 21:12:13.951118 30681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:12:13.951835 30681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:12:13.953017 30681 hybrid_clock.cc:648] HybridClock initialized: now 1786050733952988 us; error 45 us; skew 500 ppm
Aug 06 21:12:13 dist-test-slave-lgr5 krb5kdc[29773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050733, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.178.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:12:13.956032 30681 init.cc:377] Logged in from keytab as oryx/127.16.178.1@KRBTEST.COM (short username oryx)
I20260806 21:12:13.957126 30681 webserver.cc:460] Webserver started at http://127.16.178.1:34879/ using document root <none> and password file <none>
I20260806 21:12:13.957746 30681 fs_manager.cc:359] Metadata directory not provided
I20260806 21:12:13.957834 30681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:12:13.962213 30681 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 21:12:13.964597 30737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:13.966063 30681 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:12:13.966221 30681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-0/wal
uuid: "35d07d8e5bb048598b211b5d50ff137c"
format_stamp: "Formatted at 2026-08-06 21:12:12 on dist-test-slave-lgr5"
server_key: "0ef5132c0743a323e2e815e13b3ed350"
server_key_iv: "6150d147a37c7c25e2aed47d639be9bf"
server_key_version: "encryptionkey@0"
I20260806 21:12:13.966897 30681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:12:13.982203 30681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:12:13.985756 30681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:12:13.985944 30681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:12:13.986609 30681 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:12:13.987531 30681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:12:13.987582 30681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:13.987640 30681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:12:13.987668 30681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:13.996913 30681 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.1:46769
I20260806 21:12:13.996937 30901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.1:46769 every 8 connection(s)
I20260806 21:12:13.997995 30681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
I20260806 21:12:14.002421 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 30681
I20260806 21:12:14.002885 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.2:44659
--local_ip_for_outbound_sockets=127.16.178.2
--tserver_master_addrs=127.16.178.62:44553
--webserver_port=33599
--webserver_interface=127.16.178.2
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab
--principal=kudu/127.16.178.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:32925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.16.178.2
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 21:12:14 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.1@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:14 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.1@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:14.012682 30907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:14.000535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.1:60875)
0806 21:12:14.001108 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:14.001158 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:14.002475 (+ 1317us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:14.003938 (+ 1463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:14.003948 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:14.004455 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:14.004943 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:14.004958 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:14.006296 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:14.006299 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:14.007185 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:14.007198 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:14.008337 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:14.008340 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:14.008498 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:14.009136 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:14.009153 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:14.011702 (+ 2549us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":189,"threads_started":1}
W20260806 21:12:14.013201 30906 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:14.000379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:60875 (local address 127.16.178.62:44553)
0806 21:12:14.000961 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:14.000984 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:14.001285 (+ 301us) server_negotiation.cc:407] Connection header received
0806 21:12:14.003047 (+ 1762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:14.003064 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:14.003464 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:14.003809 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:14.005144 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:14.006148 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:14.007349 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:14.008163 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:14.008375 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:14.012158 (+ 3783us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.1:60875: BlockingRecv error: recv got EOF from 127.16.178.1:60875 (error 108)
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":131,"threads_started":1}
W20260806 21:12:14.013321 30903 heartbeater.cc:644] Failed to heartbeat to 127.16.178.62:44553 (0 consecutive failures): Not authorized: Failed to ping master at 127.16.178.62:44553: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Aug 06 21:12:14 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.1@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:14 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.1@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:14.019078 30907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:14.013908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.1:43157)
0806 21:12:14.014042 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:14.014057 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:14.014163 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:14.014405 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:14.014408 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:14.014572 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:14.014680 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:14.014686 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:14.015467 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:14.015471 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:14.016019 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:14.016028 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:14.016487 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:14.016491 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:14.016578 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:14.017073 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:14.017084 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:14.018935 (+ 1851us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 21:12:14.019194 30906 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:14.013983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:43157 (local address 127.16.178.62:44553)
0806 21:12:14.014093 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:14.014097 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:14.014113 (+ 16us) server_negotiation.cc:407] Connection header received
0806 21:12:14.014254 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:14.014257 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:14.014306 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:14.014372 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:14.014794 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:14.015361 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:14.016164 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:14.016385 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:14.016471 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:14.019079 (+ 2608us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.1:43157: BlockingRecv error: recv got EOF from 127.16.178.1:43157 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 06 21:12:14 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.1@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:14 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.1@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:14.025305 30907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:14.019633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.1:59825)
0806 21:12:14.019750 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:14.019763 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:14.019827 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:14.020100 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:14.020103 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:14.020226 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:14.020302 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:14.020307 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:14.021385 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:14.021388 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:14.021973 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:14.021983 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:14.022525 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:14.022528 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:14.022588 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:14.023099 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:14.023110 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:14.025182 (+ 2072us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":43}
W20260806 21:12:14.025511 30906 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:14.019717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:59825 (local address 127.16.178.62:44553)
0806 21:12:14.019853 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:14.019857 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:14.019873 (+ 16us) server_negotiation.cc:407] Connection header received
0806 21:12:14.019917 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:14.019920 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:14.019971 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:14.020070 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:14.020426 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:14.021245 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:14.022087 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:14.022416 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:14.022521 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:14.025387 (+ 2866us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.1:59825: BlockingRecv error: recv got EOF from 127.16.178.1:59825 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260806 21:12:14.025772 30903 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 21:12:14.127676 30908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:12:14.128008 30908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:12:14.128053 30908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:12:14.132501 30908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:12:14.132591 30908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:12:14.132654 30908 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.2
I20260806 21:12:14.138988 30908 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:32925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.2.keytab
--principal=oryx/127.16.178.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.2:44659
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.178.2
--webserver_port=33599
--tserver_master_addrs=127.16.178.62:44553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30908
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.2
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:12:14.140404 30908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:12:14.141548 30908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 21:12:14.153591 30908 server_base.cc:1034] running on GCE node
W20260806 21:12:14.153617 30920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:14.153617 30916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:14.153913 30917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:14.154443 30908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:12:14.155222 30908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:12:14.156440 30908 hybrid_clock.cc:648] HybridClock initialized: now 1786050734156412 us; error 42 us; skew 500 ppm
Aug 06 21:12:14 dist-test-slave-lgr5 krb5kdc[29773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050734, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.178.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:12:14.160406 30908 init.cc:377] Logged in from keytab as oryx/127.16.178.2@KRBTEST.COM (short username oryx)
I20260806 21:12:14.161638 30908 webserver.cc:460] Webserver started at http://127.16.178.2:33599/ using document root <none> and password file <none>
I20260806 21:12:14.162307 30908 fs_manager.cc:359] Metadata directory not provided
I20260806 21:12:14.162421 30908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:12:14.167714 30908 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:12:14.170400 30931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:14.171933 30908 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:12:14.172238 30908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-1/wal
uuid: "0aa0fcaf7fb0482a85970a939ea2ac2c"
format_stamp: "Formatted at 2026-08-06 21:12:13 on dist-test-slave-lgr5"
server_key: "bd300968564dac0d2e12c4c84fa74114"
server_key_iv: "4fa5167ac9a51bf9ddedba2d543aa1c5"
server_key_version: "encryptionkey@0"
I20260806 21:12:14.172945 30908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:12:14.187129 30908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:12:14.191385 30908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:12:14.191619 30908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:12:14.192389 30908 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:12:14.193562 30908 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:12:14.193635 30908 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:14.193698 30908 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:12:14.193724 30908 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:14.205015 30908 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.2:44659
I20260806 21:12:14.205047 31100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.2:44659 every 8 connection(s)
I20260806 21:12:14.206233 30908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
I20260806 21:12:14.211658 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 30908
I20260806 21:12:14.212177 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.3:34007
--local_ip_for_outbound_sockets=127.16.178.3
--tserver_master_addrs=127.16.178.62:44553
--webserver_port=35367
--webserver_interface=127.16.178.3
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab
--principal=kudu/127.16.178.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:32925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.16.178.3
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 21:12:14 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.2@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:14 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.2@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
W20260806 21:12:14.219789 30906 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:14.208404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:43241 (local address 127.16.178.62:44553)
0806 21:12:14.208597 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:14.208603 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:14.209889 (+ 1286us) server_negotiation.cc:407] Connection header received
0806 21:12:14.210820 (+ 931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:14.210822 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:14.210895 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:14.210980 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:14.212529 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:14.213273 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:14.214952 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:14.215206 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:14.215305 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:14.219661 (+ 4356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.2:43241: BlockingRecv error: recv got EOF from 127.16.178.2:43241 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 21:12:14.220875 31106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:14.208918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.2:43241)
0806 21:12:14.209700 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:14.209752 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:14.210604 (+ 852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:14.211182 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:14.211189 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:14.211605 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:14.212306 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:14.212336 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:14.213458 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:14.213461 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:14.214783 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:14.214800 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:14.215400 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:14.215405 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:14.215632 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:14.216339 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:14.216362 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:14.219450 (+ 3088us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":140,"threads_started":1}
W20260806 21:12:14.221509 31102 heartbeater.cc:644] Failed to heartbeat to 127.16.178.62:44553 (0 consecutive failures): Not authorized: Failed to ping master at 127.16.178.62:44553: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Aug 06 21:12:14 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.2@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:14 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.2@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:14.228632 31106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:14.222146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.2:34989)
0806 21:12:14.222361 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:14.222377 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:14.222476 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:14.222765 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:14.222767 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:14.222950 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:14.223071 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:14.223077 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:14.224249 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:14.224253 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:14.224865 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:14.224878 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:14.225530 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:14.225534 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:14.225618 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:14.226176 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:14.226190 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:14.228466 (+ 2276us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134}
W20260806 21:12:14.228892 30906 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:14.222287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:34989 (local address 127.16.178.62:44553)
0806 21:12:14.222452 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:14.222457 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:14.222479 (+ 22us) server_negotiation.cc:407] Connection header received
0806 21:12:14.222580 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:14.222583 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:14.222645 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:14.222750 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:14.223209 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:14.224058 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:14.225026 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:14.225380 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:14.225509 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:14.228749 (+ 3240us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.2:34989: BlockingRecv error: recv got EOF from 127.16.178.2:34989 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 06 21:12:14 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.2@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:14 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.2@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:14.235620 31106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:14.229761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.2:48263)
0806 21:12:14.229908 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:14.229924 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:14.230033 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:14.230276 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:14.230279 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:14.230464 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:14.230556 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:14.230562 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:14.231445 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:14.231449 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:14.232006 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:14.232016 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:14.232536 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:14.232540 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:14.232620 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:14.233129 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:14.233145 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:14.235471 (+ 2326us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":41}
W20260806 21:12:14.235993 31102 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 21:12:14.236043 30906 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:14.229861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:48263 (local address 127.16.178.62:44553)
0806 21:12:14.229990 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:14.229994 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:14.230010 (+ 16us) server_negotiation.cc:407] Connection header received
0806 21:12:14.230120 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:14.230124 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:14.230171 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:14.230238 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:14.230680 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:14.231314 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:14.232165 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:14.232411 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:14.232509 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:14.235902 (+ 3393us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.2:48263: BlockingRecv error: recv got EOF from 127.16.178.2:48263 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260806 21:12:14.347772 31109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:12:14.348070 31109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:12:14.348155 31109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:12:14.352272 31109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:12:14.352337 31109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:12:14.352399 31109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.3
I20260806 21:12:14.358512 31109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:32925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/krb5kdc/oryx_127.16.178.3.keytab
--principal=oryx/127.16.178.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.3:34007
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.178.3
--webserver_port=35367
--tserver_master_addrs=127.16.178.62:44553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.3
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:12:14.359722 31109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:12:14.360904 31109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 21:12:14.372148 31109 server_base.cc:1034] running on GCE node
W20260806 21:12:14.372057 31114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:14.372038 31115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:14.372038 31119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:14.372951 31109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:12:14.373793 31109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:12:14.374984 31109 hybrid_clock.cc:648] HybridClock initialized: now 1786050734374958 us; error 45 us; skew 500 ppm
Aug 06 21:12:14 dist-test-slave-lgr5 krb5kdc[29773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050734, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.178.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:12:14.378510 31109 init.cc:377] Logged in from keytab as oryx/127.16.178.3@KRBTEST.COM (short username oryx)
I20260806 21:12:14.379794 31109 webserver.cc:460] Webserver started at http://127.16.178.3:35367/ using document root <none> and password file <none>
I20260806 21:12:14.380528 31109 fs_manager.cc:359] Metadata directory not provided
I20260806 21:12:14.380631 31109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:12:14.385720 31109 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 21:12:14.388528 31131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:14.390770 31109 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 21:12:14.391037 31109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-2/wal
uuid: "ac8be99728bb4ae585127dc7665a56c7"
format_stamp: "Formatted at 2026-08-06 21:12:13 on dist-test-slave-lgr5"
server_key: "77f245680414f94fc3ba742896ecdaab"
server_key_iv: "e9bd65c94bb0437c736ac4a6a00f3114"
server_key_version: "encryptionkey@0"
I20260806 21:12:14.392154 31109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:12:14.409214 31109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:12:14.412760 31109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:12:14.412984 31109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:12:14.413704 31109 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:12:14.414719 31109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:12:14.414775 31109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:14.414836 31109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:12:14.414862 31109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:14.424808 31109 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.3:34007
I20260806 21:12:14.424839 31307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.3:34007 every 8 connection(s)
I20260806 21:12:14.425997 31109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
I20260806 21:12:14.432466 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 31109
Aug 06 21:12:14 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.3@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:14 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.3@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
W20260806 21:12:14.440022 30906 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:14.428354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:39147 (local address 127.16.178.62:44553)
0806 21:12:14.428500 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:14.428503 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:14.429976 (+ 1473us) server_negotiation.cc:407] Connection header received
0806 21:12:14.431084 (+ 1108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:14.431086 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:14.431143 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:14.431211 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:14.432708 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:14.433330 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:14.434746 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:14.434974 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:14.435086 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:14.439876 (+ 4790us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.3:39147: BlockingRecv error: recv got EOF from 127.16.178.3:39147 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 21:12:14.440766 31310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:14.428755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.3:39147)
0806 21:12:14.429792 (+ 1037us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:14.429855 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:14.430837 (+ 982us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:14.431452 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:14.431460 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:14.431872 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:14.432522 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:14.432540 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:14.433484 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:14.433487 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:14.434601 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:14.434616 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:14.435152 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:14.435157 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:14.435344 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:14.436135 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:14.436159 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:14.439710 (+ 3551us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":818,"thread_start_us":632,"threads_started":1}
W20260806 21:12:14.441524 31308 heartbeater.cc:644] Failed to heartbeat to 127.16.178.62:44553 (0 consecutive failures): Not authorized: Failed to ping master at 127.16.178.62:44553: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
I20260806 21:12:14.444625 31313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:14.433848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33090 (local address 127.16.178.62:44553)
0806 21:12:14.434162 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:14.434166 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:14.434179 (+ 13us) server_negotiation.cc:407] Connection header received
0806 21:12:14.434219 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:14.434221 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:14.434268 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:14.434365 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:14.435235 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:14.435872 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:14.437172 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:14.437616 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:14.437729 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:14.439678 (+ 1949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:14.439687 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:14.439713 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:14.439758 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:14.442013 (+ 2255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:14.442600 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:14.442609 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:14.442616 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:14.442795 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:14.443116 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:14.443120 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:14.443122 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:14.443580 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:14.443795 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:14.444189 (+ 394us) server_negotiation.cc:299] Negotiation successful
0806 21:12:14.444477 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":118,"threads_started":1}
Aug 06 21:12:14 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.3@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:14 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.3@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:14.448146 31310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:14.442056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.3:47135)
0806 21:12:14.442174 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:14.442189 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:14.442282 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:14.442512 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:14.442516 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:14.442704 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:14.442835 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:14.442843 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:14.443727 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:14.443732 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:14.444498 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:14.444515 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:14.445062 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:14.445065 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:14.445149 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:14.445622 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:14.445635 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:14.447973 (+ 2338us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 21:12:14.448339 30906 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:14.442173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:47135 (local address 127.16.178.62:44553)
0806 21:12:14.442298 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:14.442303 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:14.442319 (+ 16us) server_negotiation.cc:407] Connection header received
0806 21:12:14.442361 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:14.442363 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:14.442404 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:14.442472 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:14.442966 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:14.443498 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:14.444638 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:14.444931 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:14.445053 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:14.448227 (+ 3174us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.3:47135: BlockingRecv error: recv got EOF from 127.16.178.3:47135 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 06 21:12:14 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.3@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:14 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.3@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:14.453789 31310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:14.448930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.3:50781)
0806 21:12:14.449062 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:14.449077 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:14.449168 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:14.449381 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:14.449383 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:14.449516 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:14.449603 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:14.449607 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:14.450283 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:14.450286 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:14.450823 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:14.450831 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:14.451203 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:14.451205 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:14.451265 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:14.451736 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:14.451753 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:14.453634 (+ 1881us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 21:12:14.453919 30906 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:14.449014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:50781 (local address 127.16.178.62:44553)
0806 21:12:14.449186 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:14.449189 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:14.449203 (+ 14us) server_negotiation.cc:407] Connection header received
0806 21:12:14.449242 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:14.449244 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:14.449284 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:14.449349 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:14.449693 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:14.450198 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:14.450908 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:14.451123 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:14.451190 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:14.453815 (+ 2625us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.3:50781: BlockingRecv error: recv got EOF from 127.16.178.3:50781 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
W20260806 21:12:14.454118 31308 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 06 21:12:15 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.1@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:15 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.1@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:15.032503 31314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:15.026522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.1:37343)
0806 21:12:15.026902 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:15.026918 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:15.027049 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:15.027391 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:15.027394 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:15.027661 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:15.027819 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:15.027829 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:15.028704 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:15.028708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:15.029306 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:15.029315 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:15.029799 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:15.029803 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:15.029891 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:15.030421 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:15.030434 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:15.032342 (+ 1908us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":89,"threads_started":1}
W20260806 21:12:15.032661 31316 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:15.026605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:37343 (local address 127.16.178.62:44553)
0806 21:12:15.026906 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:15.026911 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:15.026999 (+ 88us) server_negotiation.cc:407] Connection header received
0806 21:12:15.027146 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:15.027149 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:15.027209 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:15.027308 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:15.027952 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:15.028566 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:15.029414 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:15.029697 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:15.029778 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:15.032537 (+ 2759us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.1:37343: BlockingRecv error: recv got EOF from 127.16.178.1:37343 (error 108)
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":99,"threads_started":1}
Aug 06 21:12:15 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.2@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:15 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.2@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:15.242596 31318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:15.236823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.2:58747)
0806 21:12:15.237172 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:15.237189 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:15.237331 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:15.237594 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:15.237597 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:15.237807 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:15.237907 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:15.237913 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:15.238668 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:15.238671 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:15.239240 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:15.239248 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:15.239688 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:15.239695 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:15.239778 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:15.240349 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:15.240361 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:15.242409 (+ 2048us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":123,"threads_started":1}
W20260806 21:12:15.242682 31316 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:15.236977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:58747 (local address 127.16.178.62:44553)
0806 21:12:15.237173 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:15.237177 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:15.237258 (+ 81us) server_negotiation.cc:407] Connection header received
0806 21:12:15.237430 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:15.237433 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:15.237485 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:15.237559 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:15.238030 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:15.238561 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:15.239356 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:15.239573 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:15.239661 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:15.242569 (+ 2908us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.2:58747: BlockingRecv error: recv got EOF from 127.16.178.2:58747 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 06 21:12:15 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.3@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:15 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.3@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:15.460453 31320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:15.454909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.3:49977)
0806 21:12:15.455218 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:15.455233 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:15.455347 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:15.455593 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:15.455595 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:15.455780 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:15.455875 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:15.455881 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:15.456603 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:15.456606 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:15.457181 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:15.457189 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:15.457617 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:15.457621 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:15.457702 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:15.458275 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:15.458289 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:15.460294 (+ 2005us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":97,"threads_started":1}
W20260806 21:12:15.460546 31316 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:15.455033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:49977 (local address 127.16.178.62:44553)
0806 21:12:15.455197 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:15.455201 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:15.455306 (+ 105us) server_negotiation.cc:407] Connection header received
0806 21:12:15.455433 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:15.455435 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:15.455497 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:15.455557 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:15.455964 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:15.456520 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:15.457271 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:15.457491 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:15.457561 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:15.460443 (+ 2882us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.3:49977: BlockingRecv error: recv got EOF from 127.16.178.3:49977 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 06 21:12:16 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.1@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:16 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.1@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:16.040112 31321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:16.033634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.1:58107)
0806 21:12:16.033925 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:16.033940 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:16.034062 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:16.034420 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:16.034423 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:16.034647 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:16.034755 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:16.034760 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:16.035695 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:16.035698 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:16.036502 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:16.036520 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:16.037113 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:16.037117 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:16.037213 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:16.037711 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:16.037723 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:16.039888 (+ 2165us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":82,"threads_started":1}
W20260806 21:12:16.040264 31323 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:16.033733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:58107 (local address 127.16.178.62:44553)
0806 21:12:16.034028 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:16.034032 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:16.034052 (+ 20us) server_negotiation.cc:407] Connection header received
0806 21:12:16.034152 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:16.034154 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:16.034207 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:16.034322 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:16.034921 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:16.035557 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:16.036676 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:16.036975 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:16.037079 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:16.040121 (+ 3042us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.1:58107: BlockingRecv error: recv got EOF from 127.16.178.1:58107 (error 108)
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":95,"threads_started":1}
Aug 06 21:12:16 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.2@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:16 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.2@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:16.251930 31328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:16.243552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.2:54337)
0806 21:12:16.243815 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:16.243829 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:16.243930 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:16.244166 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:16.244168 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:16.244347 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:16.244443 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:16.244448 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:16.245167 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:16.245171 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:16.245745 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:16.245754 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:16.246207 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:16.246210 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:16.246272 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:16.249706 (+ 3434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:16.249718 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:16.251718 (+ 2000us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":77,"threads_started":1}
W20260806 21:12:16.251952 31323 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:16.243657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:54337 (local address 127.16.178.62:44553)
0806 21:12:16.243785 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:16.243789 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:16.243890 (+ 101us) server_negotiation.cc:407] Connection header received
0806 21:12:16.244003 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:16.244005 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:16.244053 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:16.244153 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:16.244531 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:16.245051 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:16.245884 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:16.246110 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:16.246199 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:16.251850 (+ 5651us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.2:54337: BlockingRecv error: recv got EOF from 127.16.178.2:54337 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 06 21:12:16 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.3@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:16 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.3@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:16.467133 31330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:16.461345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.3:43671)
0806 21:12:16.461644 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:16.461663 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:16.461806 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:16.462101 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:16.462104 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:16.462324 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:16.462429 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:16.462436 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:16.463223 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:16.463227 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:16.463803 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:16.463813 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:16.464360 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:16.464364 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:16.464447 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:16.465019 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:16.465037 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:16.466975 (+ 1938us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":82,"threads_started":1}
W20260806 21:12:16.467231 31323 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:16.461549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:43671 (local address 127.16.178.62:44553)
0806 21:12:16.461689 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:16.461694 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:16.461729 (+ 35us) server_negotiation.cc:407] Connection header received
0806 21:12:16.461915 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:16.461919 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:16.461975 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:16.462062 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:16.462564 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:16.463105 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:16.463940 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:16.464236 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:16.464350 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:16.467125 (+ 2775us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.3:43671: BlockingRecv error: recv got EOF from 127.16.178.3:43671 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 06 21:12:17 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.1@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:17 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.1@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:17.046301 31331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:17.041063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.1:58283)
0806 21:12:17.041349 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:17.041364 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:17.041463 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:17.041688 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:17.041690 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:17.041866 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:17.041965 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:17.041971 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:17.042720 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:17.042723 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:17.043315 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:17.043351 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:17.043755 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:17.043757 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:17.043825 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:17.044308 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:17.044351 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:17.046163 (+ 1812us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":75,"threads_started":1}
W20260806 21:12:17.046440 31332 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:17.041126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:58283 (local address 127.16.178.62:44553)
0806 21:12:17.041382 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:17.041385 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:17.041413 (+ 28us) server_negotiation.cc:407] Connection header received
0806 21:12:17.041546 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:17.041548 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:17.041592 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:17.041665 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:17.042059 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:17.042612 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:17.043433 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:17.043659 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:17.043728 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:17.046315 (+ 2587us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.1:58283: BlockingRecv error: recv got EOF from 127.16.178.1:58283 (error 108)
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":96,"threads_started":1}
Aug 06 21:12:17 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.2@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:17 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.2@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:17.258517 31334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:17.253003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.2:42935)
0806 21:12:17.253321 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:17.253338 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:17.253486 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:17.253777 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:17.253779 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:17.254013 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:17.254140 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:17.254148 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:17.254916 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:17.254919 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:17.255491 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:17.255498 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:17.255868 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:17.255870 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:17.255926 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:17.256446 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:17.256459 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:17.258392 (+ 1933us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":84,"threads_started":1}
W20260806 21:12:17.258677 31332 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:17.253126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:42935 (local address 127.16.178.62:44553)
0806 21:12:17.253293 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:17.253298 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:17.253416 (+ 118us) server_negotiation.cc:407] Connection header received
0806 21:12:17.253593 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:17.253595 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:17.253655 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:17.253738 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:17.254263 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:17.254822 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:17.255579 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:17.255790 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:17.255858 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:17.258557 (+ 2699us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.2:42935: BlockingRecv error: recv got EOF from 127.16.178.2:42935 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 06 21:12:17 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.3@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:17 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.3@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:17.473510 31335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:17.468052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.3:60821)
0806 21:12:17.468372 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:17.468388 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:17.468523 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:17.468814 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:17.468818 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:17.469040 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:17.469143 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:17.469149 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:17.469964 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:17.469971 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:17.470509 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:17.470516 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:17.470954 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:17.470958 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:17.471037 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:17.471493 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:17.471505 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:17.473354 (+ 1849us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":87,"threads_started":1}
W20260806 21:12:17.473624 31332 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:17.468172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:60821 (local address 127.16.178.62:44553)
0806 21:12:17.468320 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:17.468324 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:17.468469 (+ 145us) server_negotiation.cc:407] Connection header received
0806 21:12:17.468623 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:17.468626 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:17.468686 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:17.468757 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:17.469257 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:17.469837 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:17.470632 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:17.470854 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:17.470941 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:17.473521 (+ 2580us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.3:60821: BlockingRecv error: recv got EOF from 127.16.178.3:60821 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 06 21:12:18 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.1@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:18 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.1@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:18.053659 31336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:18.047276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.1:53857)
0806 21:12:18.047579 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:18.047604 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:18.047742 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:18.048000 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:18.048002 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:18.048235 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:18.048381 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:18.048389 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:18.049394 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:18.049398 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:18.050174 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:18.050187 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:18.050845 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:18.050848 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:18.050924 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:18.051502 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:18.051516 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:18.053522 (+ 2006us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":80,"threads_started":1}
W20260806 21:12:18.053925 31337 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:18.047455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:53857 (local address 127.16.178.62:44553)
0806 21:12:18.047763 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:18.047767 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:18.047804 (+ 37us) server_negotiation.cc:407] Connection header received
0806 21:12:18.047849 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:18.047851 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:18.047897 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:18.047969 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:18.048596 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:18.049230 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:18.050374 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:18.050713 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:18.050818 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:18.053758 (+ 2940us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.1:53857: BlockingRecv error: recv got EOF from 127.16.178.1:53857 (error 108)
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":92,"threads_started":1}
Aug 06 21:12:18 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.2@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:18 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.2@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:18.264823 31338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:18.259296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.2:50359)
0806 21:12:18.259579 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:18.259594 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:18.259744 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:18.260007 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:18.260010 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:18.260235 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:18.260335 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:18.260341 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:18.261148 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:18.261152 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:18.261698 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:18.261706 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:18.262147 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:18.262150 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:18.262229 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:18.262750 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:18.262760 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:18.264689 (+ 1929us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":90,"threads_started":1}
W20260806 21:12:18.264999 31337 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:18.259396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:50359 (local address 127.16.178.62:44553)
0806 21:12:18.259571 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:18.259576 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:18.259679 (+ 103us) server_negotiation.cc:407] Connection header received
0806 21:12:18.259834 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:18.259837 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:18.259899 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:18.259966 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:18.260486 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:18.261024 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:18.261811 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:18.262037 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:18.262128 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:18.264849 (+ 2721us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.2:50359: BlockingRecv error: recv got EOF from 127.16.178.2:50359 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 06 21:12:18 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.3@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:18 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.3@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:18.480448 31340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:18.474457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.3:46905)
0806 21:12:18.474857 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:18.474876 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:18.474998 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:18.475320 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:18.475323 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:18.475563 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:18.475665 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:18.475671 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:18.476524 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:18.476528 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:18.477173 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:18.477185 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:18.477627 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:18.477631 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:18.477710 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:18.478241 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:18.478253 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:18.480267 (+ 2014us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":105,"threads_started":1}
W20260806 21:12:18.480571 31337 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:18.474536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:46905 (local address 127.16.178.62:44553)
0806 21:12:18.474677 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:18.474682 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:18.474946 (+ 264us) server_negotiation.cc:407] Connection header received
0806 21:12:18.475101 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:18.475104 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:18.475166 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:18.475257 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:18.475800 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:18.476415 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:18.477290 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:18.477518 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:18.477608 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:18.480457 (+ 2849us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.3:46905: BlockingRecv error: recv got EOF from 127.16.178.3:46905 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 06 21:12:19 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.1@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:19 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.1@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:19.060410 31341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:19.054763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.1:59297)
0806 21:12:19.055074 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:19.055089 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:19.055199 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:19.055485 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:19.055488 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:19.055698 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:19.055816 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:19.055823 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:19.056761 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:19.056765 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:19.057339 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:19.057347 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:19.057796 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:19.057800 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:19.057879 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:19.058381 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:19.058394 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:19.060271 (+ 1877us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":90,"threads_started":1}
W20260806 21:12:19.060551 31342 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:19.054935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:59297 (local address 127.16.178.62:44553)
0806 21:12:19.055221 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:19.055226 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:19.055244 (+ 18us) server_negotiation.cc:407] Connection header received
0806 21:12:19.055293 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:19.055295 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:19.055336 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:19.055429 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:19.055946 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:19.056644 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:19.057461 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:19.057694 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:19.057781 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:19.060424 (+ 2643us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.1:59297: BlockingRecv error: recv got EOF from 127.16.178.1:59297 (error 108)
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":91,"threads_started":1}
Aug 06 21:12:19 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.2@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:19 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.2@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:19.271507 31343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:19.265797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.2:41515)
0806 21:12:19.266119 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:19.266136 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:19.266253 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:19.266573 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:19.266576 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:19.266782 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:19.266889 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:19.266895 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:19.267673 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:19.267677 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:19.268272 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:19.268282 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:19.268739 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:19.268742 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:19.268821 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:19.269347 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:19.269359 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:19.271342 (+ 1983us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":100,"threads_started":1}
W20260806 21:12:19.271673 31342 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:19.266058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:41515 (local address 127.16.178.62:44553)
0806 21:12:19.266191 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:19.266196 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:19.266211 (+ 15us) server_negotiation.cc:407] Connection header received
0806 21:12:19.266396 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:19.266399 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:19.266450 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:19.266522 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:19.267018 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:19.267552 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:19.268399 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:19.268622 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:19.268717 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:19.271566 (+ 2849us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.2:41515: BlockingRecv error: recv got EOF from 127.16.178.2:41515 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 06 21:12:19 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.3@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:19 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.3@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:19.486795 31344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:19.481422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.3:50785)
0806 21:12:19.481723 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:19.481737 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:19.481839 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:19.482068 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:19.482070 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:19.482254 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:19.482352 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:19.482357 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:19.483072 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:19.483075 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:19.483635 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:19.483643 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:19.484171 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:19.484174 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:19.484240 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:19.484777 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:19.484789 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:19.486664 (+ 1875us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":107,"threads_started":1}
W20260806 21:12:19.486980 31342 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:19.481500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:50785 (local address 127.16.178.62:44553)
0806 21:12:19.481662 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:19.481667 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:19.481803 (+ 136us) server_negotiation.cc:407] Connection header received
0806 21:12:19.481921 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:19.481923 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:19.481971 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:19.482039 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:19.482444 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:19.482976 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:19.483766 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:19.484021 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:19.484150 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:19.486854 (+ 2704us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.3:50785: BlockingRecv error: recv got EOF from 127.16.178.3:50785 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 06 21:12:20 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.1@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:20 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.1@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:20.067579 31345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:20.061463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.1:60775)
0806 21:12:20.061771 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:20.061792 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:20.061937 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:20.062233 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:20.062236 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:20.062461 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:20.062575 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:20.062583 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:20.063416 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:20.063420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:20.064003 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:20.064013 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:20.064542 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:20.064546 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:20.064631 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:20.065252 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:20.065264 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:20.067377 (+ 2113us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":95,"threads_started":1}
W20260806 21:12:20.067704 31346 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:20.061596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:60775 (local address 127.16.178.62:44553)
0806 21:12:20.061930 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:20.061935 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:20.061963 (+ 28us) server_negotiation.cc:407] Connection header received
0806 21:12:20.062045 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:20.062048 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:20.062100 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:20.062181 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:20.062723 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:20.063287 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:20.064162 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:20.064410 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:20.064506 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:20.067548 (+ 3042us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.1:60775: BlockingRecv error: recv got EOF from 127.16.178.1:60775 (error 108)
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":123,"threads_started":1}
Aug 06 21:12:20 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.2@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:20 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.2@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:20.278126 31348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:20.272588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.2:50425)
0806 21:12:20.272950 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:20.272964 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:20.273073 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:20.273306 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:20.273308 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:20.273500 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:20.273624 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:20.273630 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:20.274414 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:20.274416 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:20.274984 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:20.274991 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:20.275469 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:20.275471 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:20.275550 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:20.276071 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:20.276124 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:20.277948 (+ 1824us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":158,"threads_started":1}
W20260806 21:12:20.278291 31346 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:20.272699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:50425 (local address 127.16.178.62:44553)
0806 21:12:20.272914 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:20.272918 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:20.273031 (+ 113us) server_negotiation.cc:407] Connection header received
0806 21:12:20.273152 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:20.273154 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:20.273201 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:20.273279 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:20.273748 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:20.274290 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:20.275110 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:20.275375 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:20.275457 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:20.278154 (+ 2697us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.2:50425: BlockingRecv error: recv got EOF from 127.16.178.2:50425 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
Aug 06 21:12:20 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.3@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:20 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.3@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:20.493049 31350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:20.487652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.3:45127)
0806 21:12:20.487954 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:20.487967 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:20.488101 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:20.488329 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:20.488331 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:20.488564 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:20.488668 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:20.488674 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:20.489392 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:20.489394 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:20.489978 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:20.489986 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:20.490375 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:20.490378 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:20.490439 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:20.490985 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:20.490997 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:20.492906 (+ 1909us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":92,"threads_started":1}
W20260806 21:12:20.493230 31346 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:20.487743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:45127 (local address 127.16.178.62:44553)
0806 21:12:20.487892 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:20.487896 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:20.488039 (+ 143us) server_negotiation.cc:407] Connection header received
0806 21:12:20.488177 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:20.488179 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:20.488226 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:20.488303 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:20.488764 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:20.489310 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:20.490073 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:20.490281 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:20.490348 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:20.493101 (+ 2753us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.3:45127: BlockingRecv error: recv got EOF from 127.16.178.3:45127 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 06 21:12:21 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.1@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:21 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.1@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:21.074296 31351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:21.068648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.1:43849)
0806 21:12:21.068969 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:21.068984 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:21.069087 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:21.069405 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:21.069406 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:21.069600 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:21.069701 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:21.069707 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:21.070478 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:21.070480 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:21.071091 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:21.071102 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:21.071613 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:21.071616 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:21.071680 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:21.072258 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:21.072271 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:21.074141 (+ 1870us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":78,"threads_started":1}
W20260806 21:12:21.074496 31352 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:21.068765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:43849 (local address 127.16.178.62:44553)
0806 21:12:21.069032 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:21.069035 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:21.069052 (+ 17us) server_negotiation.cc:407] Connection header received
0806 21:12:21.069227 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:21.069230 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:21.069284 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:21.069383 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:21.069791 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:21.070390 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:21.071206 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:21.071507 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:21.071585 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:21.074332 (+ 2747us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.1:43849: BlockingRecv error: recv got EOF from 127.16.178.1:43849 (error 108)
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":80,"threads_started":1}
Aug 06 21:12:21 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.2@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:21 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.2@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:21.289422 31355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:21.279131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.2:35665)
0806 21:12:21.279482 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:21.279498 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:21.279632 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:21.279934 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:21.279937 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:21.280190 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:21.280321 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:21.280330 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:21.281224 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:21.281226 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:21.281789 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:21.281799 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:21.282311 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:21.282316 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:21.282394 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:21.287197 (+ 4803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:21.287212 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:21.289282 (+ 2070us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":94,"threads_started":1}
W20260806 21:12:21.289665 31352 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:21.279272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:35665 (local address 127.16.178.62:44553)
0806 21:12:21.279436 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:21.279441 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:21.279594 (+ 153us) server_negotiation.cc:407] Connection header received
0806 21:12:21.279739 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:21.279742 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:21.279807 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:21.279898 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:21.280468 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:21.281093 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:21.281918 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:21.282142 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:21.282225 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:21.289533 (+ 7308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.2:35665: BlockingRecv error: recv got EOF from 127.16.178.2:35665 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 06 21:12:21 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.3@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:21 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.3@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:21.500196 31357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:21.494055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.3:59275)
0806 21:12:21.494422 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:21.494440 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:21.494578 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:21.494850 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:21.494853 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:21.495090 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:21.495189 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:21.495195 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:21.495971 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:21.495974 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:21.496643 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:21.496654 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:21.497120 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:21.497124 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:21.497206 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:21.497780 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:21.497793 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:21.500010 (+ 2217us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":98,"threads_started":1}
W20260806 21:12:21.500283 31352 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:21.494223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:59275 (local address 127.16.178.62:44553)
0806 21:12:21.494422 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:21.494428 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:21.494525 (+ 97us) server_negotiation.cc:407] Connection header received
0806 21:12:21.494676 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:21.494679 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:21.494729 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:21.494798 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:21.495326 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:21.495859 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:21.496759 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:21.496987 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:21.497078 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:21.500178 (+ 3100us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.3:59275: BlockingRecv error: recv got EOF from 127.16.178.3:59275 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 06 21:12:22 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.1@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:22 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.1@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:22.081065 31359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:22.075252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.1:57281)
0806 21:12:22.075572 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:22.075592 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:22.075717 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:22.075962 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:22.075964 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:22.076175 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:22.076304 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:22.076313 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:22.077105 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:22.077107 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:22.077758 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:22.077770 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:22.078191 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:22.078193 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:22.078253 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:22.078771 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:22.078783 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:22.080942 (+ 2159us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":98,"threads_started":1}
W20260806 21:12:22.081187 31360 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:22.075454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:57281 (local address 127.16.178.62:44553)
0806 21:12:22.075737 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:22.075740 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:22.075757 (+ 17us) server_negotiation.cc:407] Connection header received
0806 21:12:22.075804 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:22.075806 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:22.075851 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:22.075932 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:22.076433 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:22.076993 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:22.077868 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:22.078098 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:22.078179 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:22.081052 (+ 2873us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.1:57281: BlockingRecv error: recv got EOF from 127.16.178.1:57281 (error 108)
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":91,"threads_started":1}
Aug 06 21:12:22 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.2@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:22 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.2@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:22.296067 31363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:22.290375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.2:47769)
0806 21:12:22.290699 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:22.290715 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:22.290851 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:22.291145 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:22.291149 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:22.291363 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:22.291477 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:22.291484 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:22.292403 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:22.292406 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:22.292991 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:22.293001 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:22.293427 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:22.293429 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:22.293487 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:22.293989 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:22.294000 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:22.295937 (+ 1937us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":106,"threads_started":1}
W20260806 21:12:22.296265 31360 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:22.290454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:47769 (local address 127.16.178.62:44553)
0806 21:12:22.290598 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:22.290602 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:22.290802 (+ 200us) server_negotiation.cc:407] Connection header received
0806 21:12:22.290958 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:22.290960 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:22.291012 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:22.291077 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:22.291585 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:22.292292 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:22.293110 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:22.293341 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:22.293416 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:22.296128 (+ 2712us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.2:47769: BlockingRecv error: recv got EOF from 127.16.178.2:47769 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 06 21:12:22 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.3@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:22 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.3@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:22.506894 31365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:22.501140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.3:51573)
0806 21:12:22.501416 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:22.501433 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:22.501569 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:22.501860 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:22.501863 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:22.502057 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:22.502157 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:22.502163 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:22.503174 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:22.503177 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:22.503908 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:22.503920 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:22.504427 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:22.504430 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:22.504505 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:22.505046 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:22.505060 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:22.506773 (+ 1713us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":74,"threads_started":1}
W20260806 21:12:22.507058 31360 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:22.501255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:51573 (local address 127.16.178.62:44553)
0806 21:12:22.501419 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:22.501423 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:22.501557 (+ 134us) server_negotiation.cc:407] Connection header received
0806 21:12:22.501654 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:22.501656 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:22.501714 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:22.501812 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:22.502335 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:22.503042 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:22.504027 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:22.504284 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:22.504376 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:22.506931 (+ 2555us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.3:51573: BlockingRecv error: recv got EOF from 127.16.178.3:51573 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 06 21:12:23 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.1@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:23 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.1@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:23.088656 31366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:23.082081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.1:32943)
0806 21:12:23.082479 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:23.082496 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:23.082616 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:23.082919 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:23.082922 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:23.083115 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:23.083221 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:23.083227 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:23.084167 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:23.084170 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:23.084764 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:23.084777 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:23.085476 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:23.085479 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:23.085572 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:23.086272 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:23.086296 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:23.088520 (+ 2224us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":108,"threads_started":1}
W20260806 21:12:23.088889 31368 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:23.082259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:32943 (local address 127.16.178.62:44553)
0806 21:12:23.082560 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:23.082564 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:23.082582 (+ 18us) server_negotiation.cc:407] Connection header received
0806 21:12:23.082712 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:23.082714 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:23.082764 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:23.082848 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:23.083395 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:23.083970 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:23.084956 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:23.085302 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:23.085436 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:23.088740 (+ 3304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.1:32943: BlockingRecv error: recv got EOF from 127.16.178.1:32943 (error 108)
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":107,"threads_started":1}
Aug 06 21:12:23 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.2@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:23 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.2@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:23.302886 31369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:23.296896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.2:53543)
0806 21:12:23.297403 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:23.297418 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:23.297548 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:23.297852 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:23.297856 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:23.298112 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:23.298228 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:23.298235 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:23.299006 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:23.299010 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:23.299576 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:23.299585 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:23.300048 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:23.300052 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:23.300170 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:23.300735 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:23.300748 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:23.302689 (+ 1941us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":102,"threads_started":1}
W20260806 21:12:23.303052 31368 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:23.297084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:53543 (local address 127.16.178.62:44553)
0806 21:12:23.297329 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:23.297335 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:23.297559 (+ 224us) server_negotiation.cc:407] Connection header received
0806 21:12:23.297619 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:23.297622 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:23.297676 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:23.297754 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:23.298338 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:23.298890 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:23.299689 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:23.299930 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:23.300019 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:23.302904 (+ 2885us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.2:53543: BlockingRecv error: recv got EOF from 127.16.178.2:53543 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Aug 06 21:12:23 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.3@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:23 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.3@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:23.514072 31371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:23.507934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.3:55255)
0806 21:12:23.508293 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:23.508310 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:23.508445 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:23.508719 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:23.508722 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:23.508938 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:23.509062 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:23.509069 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:23.509876 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:23.509880 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:23.510449 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:23.510458 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:23.510972 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:23.510976 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:23.511064 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:23.511634 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:23.511646 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:23.513896 (+ 2250us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":133,"threads_started":1}
W20260806 21:12:23.514209 31368 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:23.508159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:55255 (local address 127.16.178.62:44553)
0806 21:12:23.508294 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:23.508299 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:23.508376 (+ 77us) server_negotiation.cc:407] Connection header received
0806 21:12:23.508554 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:23.508557 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:23.508607 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:23.508676 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:23.509199 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:23.509755 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:23.510577 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:23.510811 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:23.510908 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:23.514098 (+ 3190us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.3:55255: BlockingRecv error: recv got EOF from 127.16.178.3:55255 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 06 21:12:24 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.1@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:24 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.1@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:24.094872 31372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:24.089540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.1:46563)
0806 21:12:24.089828 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:24.089851 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:24.089936 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:24.090168 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:24.090169 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:24.090354 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:24.090455 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:24.090460 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:24.091163 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:24.091165 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:24.091703 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:24.091709 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:24.092227 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:24.092230 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:24.092297 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:24.092794 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:24.092806 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:24.094725 (+ 1919us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":92,"threads_started":1}
W20260806 21:12:24.095000 31373 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:24.089621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:46563 (local address 127.16.178.62:44553)
0806 21:12:24.089881 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:24.089885 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:24.089898 (+ 13us) server_negotiation.cc:407] Connection header received
0806 21:12:24.090022 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:24.090024 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:24.090069 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:24.090142 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:24.090547 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:24.091082 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:24.091844 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:24.092115 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:24.092210 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:24.094864 (+ 2654us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.1:46563: BlockingRecv error: recv got EOF from 127.16.178.1:46563 (error 108)
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":66,"threads_started":1}
Aug 06 21:12:24 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.2@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:24 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.2@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:24.308944 31374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:24.303748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.2:44491)
0806 21:12:24.304008 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:24.304021 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:24.304149 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:24.304371 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:24.304373 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:24.304582 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:24.304688 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:24.304694 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:24.305400 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:24.305403 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:24.305974 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:24.305983 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:24.306399 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:24.306401 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:24.306475 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:24.306943 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:24.306954 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:24.308806 (+ 1852us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":77,"threads_started":1}
W20260806 21:12:24.309098 31373 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:24.303824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:44491 (local address 127.16.178.62:44553)
0806 21:12:24.303979 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:24.303983 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:24.304102 (+ 119us) server_negotiation.cc:407] Connection header received
0806 21:12:24.304227 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:24.304229 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:24.304273 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:24.304345 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:24.304780 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:24.305305 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:24.306092 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:24.306313 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:24.306390 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:24.308964 (+ 2574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.2:44491: BlockingRecv error: recv got EOF from 127.16.178.2:44491 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 06 21:12:24 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.3@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:24 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.3@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:24.520938 31376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:24.515120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.3:57395)
0806 21:12:24.515550 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:24.515564 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:24.515683 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:24.515955 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:24.515958 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:24.516216 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:24.516335 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:24.516342 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:24.517107 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:24.517110 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:24.517674 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:24.517683 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:24.518160 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:24.518164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:24.518250 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:24.518759 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:24.518772 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:24.520788 (+ 2016us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":106,"threads_started":1}
W20260806 21:12:24.521086 31373 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:24.515330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:57395 (local address 127.16.178.62:44553)
0806 21:12:24.515494 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:24.515500 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:24.515651 (+ 151us) server_negotiation.cc:407] Connection header received
0806 21:12:24.515789 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:24.515792 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:24.515844 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:24.515909 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:24.516455 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:24.516987 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:24.517790 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:24.518027 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:24.518118 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:24.520956 (+ 2838us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.3:57395: BlockingRecv error: recv got EOF from 127.16.178.3:57395 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 06 21:12:25 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.1@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:25 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.1@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:25.102393 31377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:25.095962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.1:43335)
0806 21:12:25.096311 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:25.096325 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:25.096463 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:25.096818 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:25.096820 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:25.097029 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:25.097145 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:25.097151 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:25.098112 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:25.098117 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:25.098854 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:25.098869 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:25.099459 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:25.099463 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:25.099550 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:25.100161 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:25.100178 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:25.102194 (+ 2016us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":141,"threads_started":1}
W20260806 21:12:25.102494 31378 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:25.096172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:43335 (local address 127.16.178.62:44553)
0806 21:12:25.096522 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:25.096528 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:25.096553 (+ 25us) server_negotiation.cc:407] Connection header received
0806 21:12:25.096617 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:25.096621 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:25.096673 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:25.096803 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:25.097266 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:25.097951 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:25.099019 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:25.099335 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:25.099455 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:25.102366 (+ 2911us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.1:43335: BlockingRecv error: recv got EOF from 127.16.178.1:43335 (error 108)
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":122,"threads_started":1}
Aug 06 21:12:25 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.2@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:25 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.2@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:25.315428 31380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:25.309833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.2:40393)
0806 21:12:25.310147 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:25.310161 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:25.310280 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:25.310537 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:25.310540 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:25.310760 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:25.310863 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:25.310869 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:25.311664 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:25.311668 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:25.312311 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:25.312322 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:25.312783 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:25.312787 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:25.312865 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:25.313438 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:25.313452 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:25.315218 (+ 1766us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":97,"threads_started":1}
W20260806 21:12:25.315505 31378 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:25.309954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:40393 (local address 127.16.178.62:44553)
0806 21:12:25.310118 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:25.310124 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:25.310232 (+ 108us) server_negotiation.cc:407] Connection header received
0806 21:12:25.310371 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:25.310374 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:25.310430 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:25.310499 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:25.310986 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:25.311548 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:25.312439 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:25.312670 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:25.312764 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:25.315393 (+ 2629us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.2:40393: BlockingRecv error: recv got EOF from 127.16.178.2:40393 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 06 21:12:25 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.3@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:25 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.3@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:25.527508 31381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:25.521850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.3:59059)
0806 21:12:25.522224 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:25.522239 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:25.522383 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:25.522693 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:25.522696 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:25.522922 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:25.523042 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:25.523050 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:25.523969 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:25.523972 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:25.524593 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:25.524604 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:25.525015 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:25.525017 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:25.525074 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:25.525605 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:25.525617 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:25.527377 (+ 1760us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":117,"threads_started":1}
W20260806 21:12:25.527649 31378 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:25.521999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:59059 (local address 127.16.178.62:44553)
0806 21:12:25.522139 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:25.522144 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:25.522338 (+ 194us) server_negotiation.cc:407] Connection header received
0806 21:12:25.522497 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:25.522501 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:25.522561 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:25.522651 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:25.523187 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:25.523831 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:25.524707 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:25.524929 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:25.525004 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:25.527534 (+ 2530us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.3:59059: BlockingRecv error: recv got EOF from 127.16.178.3:59059 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 06 21:12:26 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.1@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:26 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.1@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:26.109127 31383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:26.103304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.1:33139)
0806 21:12:26.103645 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:26.103660 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:26.103760 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:26.104052 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:26.104055 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:26.104285 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:26.104420 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:26.104427 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:26.105252 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:26.105255 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:26.105831 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:26.105841 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:26.106326 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:26.106329 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:26.106426 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:26.107010 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:26.107023 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:26.109000 (+ 1977us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":118,"threads_started":1}
W20260806 21:12:26.109340 31389 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:26.103433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:33139 (local address 127.16.178.62:44553)
0806 21:12:26.103692 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:26.103696 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:26.103711 (+ 15us) server_negotiation.cc:407] Connection header received
0806 21:12:26.103848 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:26.103850 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:26.103900 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:26.103980 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:26.104565 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:26.105141 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:26.105964 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:26.106204 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:26.106295 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:26.109188 (+ 2893us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.1:33139: BlockingRecv error: recv got EOF from 127.16.178.1:33139 (error 108)
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":68,"threads_started":1}
Aug 06 21:12:26 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.2@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:26 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.2@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:26.326623 31390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:26.316414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.2:36587)
0806 21:12:26.316703 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:26.316717 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:26.316827 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:26.317060 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:26.317063 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:26.317247 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:26.317350 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:26.317355 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:26.318125 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:26.318128 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:26.318731 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:26.318741 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:26.319164 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:26.319166 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:26.319225 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:26.324603 (+ 5378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:26.324620 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:26.326493 (+ 1873us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":94,"threads_started":1}
W20260806 21:12:26.326782 31389 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:26.316550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:36587 (local address 127.16.178.62:44553)
0806 21:12:26.316713 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:26.316716 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:26.316790 (+ 74us) server_negotiation.cc:407] Connection header received
0806 21:12:26.316905 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:26.316907 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:26.316955 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:26.317037 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:26.317458 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:26.318003 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:26.318841 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:26.319074 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:26.319152 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:26.326663 (+ 7511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.2:36587: BlockingRecv error: recv got EOF from 127.16.178.2:36587 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 06 21:12:26 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.3@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:26 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.3@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:26.534451 31391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:26.528718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.3:58665)
0806 21:12:26.529094 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:26.529110 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:26.529235 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:26.529564 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:26.529567 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:26.529826 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:26.529936 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:26.529942 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:26.530830 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:26.530842 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:26.531403 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:26.531411 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:26.531827 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:26.531830 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:26.531907 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:26.532434 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:26.532446 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:26.534302 (+ 1856us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":104,"threads_started":1}
W20260806 21:12:26.534605 31389 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:26.528792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:58665 (local address 127.16.178.62:44553)
0806 21:12:26.528933 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:26.528937 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:26.529184 (+ 247us) server_negotiation.cc:407] Connection header received
0806 21:12:26.529343 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:26.529345 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:26.529398 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:26.529476 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:26.530048 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:26.530702 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:26.531512 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:26.531724 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:26.531796 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:26.534475 (+ 2679us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.3:58665: BlockingRecv error: recv got EOF from 127.16.178.3:58665 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 06 21:12:27 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.1@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:27 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.1@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:27.115464 31392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:27.110131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.1:45911)
0806 21:12:27.110428 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:27.110444 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:27.110555 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:27.110776 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:27.110777 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:27.110965 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:27.111079 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:27.111084 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:27.111840 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:27.111842 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:27.112459 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:27.112467 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:27.112938 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:27.112942 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:27.113030 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:27.113533 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:27.113546 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:27.115306 (+ 1760us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":85,"threads_started":1}
W20260806 21:12:27.115595 31395 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:27.110253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:45911 (local address 127.16.178.62:44553)
0806 21:12:27.110549 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:27.110553 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:27.110569 (+ 16us) server_negotiation.cc:407] Connection header received
0806 21:12:27.110620 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:27.110622 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:27.110667 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:27.110750 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:27.111172 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:27.111745 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:27.112596 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:27.112830 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:27.112923 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:27.115462 (+ 2539us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.1:45911: BlockingRecv error: recv got EOF from 127.16.178.1:45911 (error 108)
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":106,"threads_started":1}
Aug 06 21:12:27 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.2@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:27 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.2@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:27.332991 31398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:27.327522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.2:46593)
0806 21:12:27.327856 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:27.327870 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:27.327975 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:27.328260 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:27.328263 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:27.328551 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:27.328658 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:27.328664 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:27.329424 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:27.329427 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:27.330072 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:27.330084 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:27.330551 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:27.330554 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:27.330628 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:27.331123 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:27.331133 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:27.332865 (+ 1732us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":76,"threads_started":1}
W20260806 21:12:27.333129 31395 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:27.327741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:46593 (local address 127.16.178.62:44553)
0806 21:12:27.327887 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:27.327892 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:27.327912 (+ 20us) server_negotiation.cc:407] Connection header received
0806 21:12:27.328067 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:27.328070 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:27.328145 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:27.328229 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:27.328760 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:27.329309 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:27.330221 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:27.330449 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:27.330550 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:27.333011 (+ 2461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.2:46593: BlockingRecv error: recv got EOF from 127.16.178.2:46593 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 06 21:12:27 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.3@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:27 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.3@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:27.542238 31399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:27.535375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.3:36793)
0806 21:12:27.535663 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:27.535678 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:27.535800 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:27.536070 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:27.536072 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:27.536318 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:27.536471 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:27.536480 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:27.537529 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:27.537533 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:27.538284 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:27.538295 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:27.539039 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:27.539043 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:27.539131 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:27.539724 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:27.539740 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:27.542055 (+ 2315us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":89,"threads_started":1}
W20260806 21:12:27.544651 31395 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:27.535475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:36793 (local address 127.16.178.62:44553)
0806 21:12:27.535643 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:27.535648 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:27.535761 (+ 113us) server_negotiation.cc:407] Connection header received
0806 21:12:27.535898 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:27.535901 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:27.535952 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:27.536026 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:27.536604 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:27.537408 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:27.538402 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:27.538771 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:27.538877 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:27.544525 (+ 5648us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.3:36793: BlockingRecv error: recv got EOF from 127.16.178.3:36793 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 06 21:12:28 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.1@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:28 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.1@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:28.121951 31402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:28.116512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.1:41415)
0806 21:12:28.116814 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:28.116828 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:28.116936 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:28.117193 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:28.117195 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:28.117376 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:28.117475 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:28.117480 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:28.118228 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:28.118230 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:28.118778 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:28.118783 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:28.119178 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:28.119181 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:28.119251 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:28.119815 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:28.119830 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:28.121826 (+ 1996us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":95,"threads_started":1}
W20260806 21:12:28.122098 31403 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:28.116643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:41415 (local address 127.16.178.62:44553)
0806 21:12:28.116956 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:28.116959 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:28.116998 (+ 39us) server_negotiation.cc:407] Connection header received
0806 21:12:28.117044 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:28.117046 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:28.117092 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:28.117163 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:28.117595 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:28.118147 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:28.118868 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:28.119091 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:28.119183 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:28.121964 (+ 2781us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.1:41415: BlockingRecv error: recv got EOF from 127.16.178.1:41415 (error 108)
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":70,"threads_started":1}
Aug 06 21:12:28 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.2@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:28 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.2@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:28.339269 31404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:28.333867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.2:58075)
0806 21:12:28.334253 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:28.334269 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:28.334395 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:28.334688 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:28.334692 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:28.334899 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:28.335006 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:28.335038 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:28.335766 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:28.335769 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:28.336423 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:28.336434 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:28.336816 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:28.336819 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:28.336879 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:28.337372 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:28.337383 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:28.339133 (+ 1750us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":114,"threads_started":1}
W20260806 21:12:28.339402 31403 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:28.333907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:58075 (local address 127.16.178.62:44553)
0806 21:12:28.334072 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:28.334075 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:28.334381 (+ 306us) server_negotiation.cc:407] Connection header received
0806 21:12:28.334485 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:28.334487 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:28.334550 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:28.334642 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:28.335132 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:28.335666 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:28.336517 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:28.336732 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:28.336805 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:28.339289 (+ 2484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.2:58075: BlockingRecv error: recv got EOF from 127.16.178.2:58075 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 06 21:12:28 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.3@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:28 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.3@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:28.550259 31406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:28.544740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.3:50363)
0806 21:12:28.545050 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:28.545064 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:28.545188 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:28.545439 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:28.545442 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:28.545656 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:28.545755 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:28.545761 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:28.546528 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:28.546531 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:28.547090 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:28.547098 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:28.547556 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:28.547560 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:28.547644 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:28.548195 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:28.548207 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:28.550107 (+ 1900us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":89,"threads_started":1}
W20260806 21:12:28.550351 31403 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:28.544859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:50363 (local address 127.16.178.62:44553)
0806 21:12:28.545032 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:28.545037 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:28.545137 (+ 100us) server_negotiation.cc:407] Connection header received
0806 21:12:28.545278 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:28.545281 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:28.545331 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:28.545398 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:28.545881 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:28.546415 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:28.547207 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:28.547427 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:28.547526 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:28.550249 (+ 2723us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.3:50363: BlockingRecv error: recv got EOF from 127.16.178.3:50363 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 06 21:12:29 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.1@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:29 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.1@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:29.129122 31412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:29.122948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.1:41803)
0806 21:12:29.123300 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:29.123314 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:29.123431 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:29.123777 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:29.123779 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:29.124002 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:29.124146 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:29.124156 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:29.124984 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:29.124987 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:29.125607 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:29.125618 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:29.126112 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:29.126116 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:29.126194 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:29.126815 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:29.126830 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:29.128935 (+ 2105us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":93,"threads_started":1}
W20260806 21:12:29.129299 31413 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:29.123028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:41803 (local address 127.16.178.62:44553)
0806 21:12:29.123307 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:29.123312 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:29.123516 (+ 204us) server_negotiation.cc:407] Connection header received
0806 21:12:29.123570 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:29.123573 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:29.123629 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:29.123722 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:29.124267 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:29.124833 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:29.125719 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:29.125967 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:29.126049 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:29.129143 (+ 3094us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.1:41803: BlockingRecv error: recv got EOF from 127.16.178.1:41803 (error 108)
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":88,"threads_started":1}
Aug 06 21:12:29 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.2@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
Aug 06 21:12:29 dist-test-slave-lgr5 krb5kdc[29773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.178.2@KRBTEST.COM for oryx/127.16.178.62@KRBTEST.COM, Server not found in Kerberos database
I20260806 21:12:29.345922 31415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:29.340247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:44553 (local address 127.16.178.2:35239)
0806 21:12:29.340575 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:29.340589 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:29.340708 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:29.340961 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:29.340964 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:29.341181 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:29.341279 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:29.341285 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:29.342073 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:29.342077 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:29.342661 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:29.342670 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:29.343150 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:29.343154 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:29.343232 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:29.343785 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:29.343797 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:29.345741 (+ 1944us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.178.62:44553: Server oryx/127.16.178.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":107,"threads_started":1}
W20260806 21:12:29.346064 31413 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:29.340341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:35239 (local address 127.16.178.62:44553)
0806 21:12:29.340504 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:29.340509 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:29.340661 (+ 152us) server_negotiation.cc:407] Connection header received
0806 21:12:29.340803 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:29.340805 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:29.340857 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:29.340923 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:29.341401 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:29.341943 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:29.342788 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:29.343035 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:29.343130 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:29.345943 (+ 2813us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.178.2:35239: BlockingRecv error: recv got EOF from 127.16.178.2:35239 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 21:12:29.434466 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 30681
I20260806 21:12:29.449304 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 30908
I20260806 21:12:29.465636 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 31109
I20260806 21:12:29.490080 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 30570
2026-08-06T21:12:29Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18328 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-06T21:12:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:12:29Z Disabled control of system clock
I20260806 21:12:29.523753 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:41567
--webserver_interface=127.16.178.62
--webserver_port=0
--builtin_ntp_servers=127.16.178.20:43961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:41567
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 21:12:29.637228 31420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:12:29.637535 31420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:12:29.637569 31420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:12:29.641721 31420 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:12:29.641783 31420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:12:29.641798 31420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:12:29.641813 31420 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:12:29.641837 31420 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:12:29.646696 31420 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:43961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:41567
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:41567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.178.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:12:29.647847 31420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:12:29.649331 31420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:12:29.658157 31425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:29.659029 31420 server_base.cc:1034] running on GCE node
W20260806 21:12:29.659278 31431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:29.659658 31427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:29.660274 31420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:12:29.661434 31420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:12:29.662626 31420 hybrid_clock.cc:648] HybridClock initialized: now 1786050749662609 us; error 26 us; skew 500 ppm
I20260806 21:12:29.664534 31420 webserver.cc:460] Webserver started at http://127.16.178.62:41735/ using document root <none> and password file <none>
I20260806 21:12:29.665133 31420 fs_manager.cc:359] Metadata directory not provided
I20260806 21:12:29.665211 31420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:12:29.665408 31420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:12:29.667538 31420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "a2f9fe892ffd4dbcb401383a9a11836c"
format_stamp: "Formatted at 2026-08-06 21:12:29 on dist-test-slave-lgr5"
server_key: "6238ee575e3ae28d22eb5ae4d4a17a7b"
server_key_iv: "faebf002a5f7c12a0bf3ef253ac4cc7f"
server_key_version: "encryptionkey@0"
I20260806 21:12:29.668159 31420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "a2f9fe892ffd4dbcb401383a9a11836c"
format_stamp: "Formatted at 2026-08-06 21:12:29 on dist-test-slave-lgr5"
server_key: "6238ee575e3ae28d22eb5ae4d4a17a7b"
server_key_iv: "faebf002a5f7c12a0bf3ef253ac4cc7f"
server_key_version: "encryptionkey@0"
I20260806 21:12:29.672564 31420 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 21:12:29.675354 31441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:29.676741 31420 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:12:29.676877 31420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "a2f9fe892ffd4dbcb401383a9a11836c"
format_stamp: "Formatted at 2026-08-06 21:12:29 on dist-test-slave-lgr5"
server_key: "6238ee575e3ae28d22eb5ae4d4a17a7b"
server_key_iv: "faebf002a5f7c12a0bf3ef253ac4cc7f"
server_key_version: "encryptionkey@0"
I20260806 21:12:29.677003 31420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:12:29.698395 31420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:12:29.699198 31420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:12:29.699389 31420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:12:29.706188 31420 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.62:41567
I20260806 21:12:29.706211 31524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.62:41567 every 8 connection(s)
I20260806 21:12:29.707140 31420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:12:29.710093 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 31420
I20260806 21:12:29.710206 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:12:29.710393 31525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:12:29.710501 17096 external_mini_cluster.cc:1428] Setting key 4812c47d7410c8a708c170cefe8b5051
I20260806 21:12:29.716748 31525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2f9fe892ffd4dbcb401383a9a11836c: Bootstrap starting.
I20260806 21:12:29.719106 31525 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2f9fe892ffd4dbcb401383a9a11836c: Neither blocks nor log segments found. Creating new log.
I20260806 21:12:29.719568 31529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:29.711896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33862 (local address 127.16.178.62:41567)
0806 21:12:29.712555 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:29.712571 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:29.712605 (+ 34us) server_negotiation.cc:407] Connection header received
0806 21:12:29.713253 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:29.713270 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:29.713644 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:29.713944 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:29.714376 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:29.715387 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:29.716357 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:29.716832 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:29.716993 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:29.717857 (+ 864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:29.717862 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:29.717874 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:12:29.718253 (+ 379us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:29.718311 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:12:29.718331 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:29.718418 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:29.718560 (+ 142us) server_negotiation.cc:299] Negotiation successful
0806 21:12:29.718810 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":151,"threads_started":1}
I20260806 21:12:29.719972 31525 log.cc:824] T 00000000000000000000000000000000 P a2f9fe892ffd4dbcb401383a9a11836c: Log is configured to *not* fsync() on all Append() calls
I20260806 21:12:29.721943 31525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2f9fe892ffd4dbcb401383a9a11836c: No bootstrap required, opened a new log
I20260806 21:12:29.725243 31525 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2f9fe892ffd4dbcb401383a9a11836c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2f9fe892ffd4dbcb401383a9a11836c" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 41567 } }
I20260806 21:12:29.725456 31525 raft_consensus.cc:374] T 00000000000000000000000000000000 P a2f9fe892ffd4dbcb401383a9a11836c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:12:29.725497 31525 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2f9fe892ffd4dbcb401383a9a11836c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2f9fe892ffd4dbcb401383a9a11836c, State: Initialized, Role: FOLLOWER
I20260806 21:12:29.725847 31525 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2f9fe892ffd4dbcb401383a9a11836c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2f9fe892ffd4dbcb401383a9a11836c" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 41567 } }
I20260806 21:12:29.725953 31525 raft_consensus.cc:388] T 00000000000000000000000000000000 P a2f9fe892ffd4dbcb401383a9a11836c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:12:29.725996 31525 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2f9fe892ffd4dbcb401383a9a11836c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:12:29.726079 31525 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2f9fe892ffd4dbcb401383a9a11836c [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:12:29.727069 31525 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2f9fe892ffd4dbcb401383a9a11836c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2f9fe892ffd4dbcb401383a9a11836c" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 41567 } }
I20260806 21:12:29.727372 31525 leader_election.cc:302] T 00000000000000000000000000000000 P a2f9fe892ffd4dbcb401383a9a11836c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2f9fe892ffd4dbcb401383a9a11836c; no voters:
I20260806 21:12:29.727624 31525 leader_election.cc:288] T 00000000000000000000000000000000 P a2f9fe892ffd4dbcb401383a9a11836c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:12:29.727726 31533 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a2f9fe892ffd4dbcb401383a9a11836c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:12:29.727955 31533 raft_consensus.cc:686] T 00000000000000000000000000000000 P a2f9fe892ffd4dbcb401383a9a11836c [term 1 LEADER]: Becoming Leader. State: Replica: a2f9fe892ffd4dbcb401383a9a11836c, State: Running, Role: LEADER
I20260806 21:12:29.728320 31533 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2f9fe892ffd4dbcb401383a9a11836c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2f9fe892ffd4dbcb401383a9a11836c" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 41567 } }
I20260806 21:12:29.728809 31525 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2f9fe892ffd4dbcb401383a9a11836c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:12:29.729866 31535 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2f9fe892ffd4dbcb401383a9a11836c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a2f9fe892ffd4dbcb401383a9a11836c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2f9fe892ffd4dbcb401383a9a11836c" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 41567 } } }
I20260806 21:12:29.730000 31535 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2f9fe892ffd4dbcb401383a9a11836c [sys.catalog]: This master's current role is: LEADER
I20260806 21:12:29.730281 31536 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2f9fe892ffd4dbcb401383a9a11836c [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2f9fe892ffd4dbcb401383a9a11836c. Latest consensus state: current_term: 1 leader_uuid: "a2f9fe892ffd4dbcb401383a9a11836c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2f9fe892ffd4dbcb401383a9a11836c" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 41567 } } }
I20260806 21:12:29.730378 31536 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2f9fe892ffd4dbcb401383a9a11836c [sys.catalog]: This master's current role is: LEADER
I20260806 21:12:29.730382 31551 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:12:29.733413 31551 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:12:29.739090 31551 catalog_manager.cc:1347] Generated new cluster ID: 71684c2f24e941bcaaf7b1b73f3c0b1b
I20260806 21:12:29.739154 31551 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:12:29.755754 31551 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:12:29.756764 31551 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:12:29.761991 31551 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a2f9fe892ffd4dbcb401383a9a11836c: Generated new TSK 0
I20260806 21:12:29.762632 31551 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:12:29.765364 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.1:0
--local_ip_for_outbound_sockets=127.16.178.1
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:41567
--builtin_ntp_servers=127.16.178.20:43961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 21:12:29.880882 31567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:12:29.881186 31567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:12:29.881244 31567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:12:29.885221 31567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:12:29.885272 31567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:12:29.885324 31567 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.1
I20260806 21:12:29.890247 31567 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:43961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:41567
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.1
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:12:29.891306 31567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:12:29.892736 31567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:12:29.902594 31577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:29.902590 31582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:29.902839 31567 server_base.cc:1034] running on GCE node
W20260806 21:12:29.902598 31578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:29.903406 31567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:12:29.904107 31567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:12:29.905292 31567 hybrid_clock.cc:648] HybridClock initialized: now 1786050749905273 us; error 31 us; skew 500 ppm
I20260806 21:12:29.907387 31567 webserver.cc:460] Webserver started at http://127.16.178.1:39365/ using document root <none> and password file <none>
I20260806 21:12:29.908002 31567 fs_manager.cc:359] Metadata directory not provided
I20260806 21:12:29.908118 31567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:12:29.908361 31567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:12:29.910558 31567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/data/instance:
uuid: "dc4060b6d9c4403c81bda82dc8f5be38"
format_stamp: "Formatted at 2026-08-06 21:12:29 on dist-test-slave-lgr5"
server_key: "4673c3a1b60d90d68d2119779da30b82"
server_key_iv: "c035d3917ea3c864358cb2a4d9fbf305"
server_key_version: "encryptionkey@0"
I20260806 21:12:29.911150 31567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance:
uuid: "dc4060b6d9c4403c81bda82dc8f5be38"
format_stamp: "Formatted at 2026-08-06 21:12:29 on dist-test-slave-lgr5"
server_key: "4673c3a1b60d90d68d2119779da30b82"
server_key_iv: "c035d3917ea3c864358cb2a4d9fbf305"
server_key_version: "encryptionkey@0"
I20260806 21:12:29.915496 31567 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260806 21:12:29.918474 31593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:29.919802 31567 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:12:29.919937 31567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/wal
uuid: "dc4060b6d9c4403c81bda82dc8f5be38"
format_stamp: "Formatted at 2026-08-06 21:12:29 on dist-test-slave-lgr5"
server_key: "4673c3a1b60d90d68d2119779da30b82"
server_key_iv: "c035d3917ea3c864358cb2a4d9fbf305"
server_key_version: "encryptionkey@0"
I20260806 21:12:29.920055 31567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:12:29.937175 31567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:12:29.937850 31567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:12:29.938019 31567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:12:29.938616 31567 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:12:29.939558 31567 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:12:29.939605 31567 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:29.939648 31567 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:12:29.939677 31567 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:29.947922 31567 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.1:39927
I20260806 21:12:29.947983 31763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.1:39927 every 8 connection(s)
I20260806 21:12:29.948838 31567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
I20260806 21:12:29.952404 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 31567
I20260806 21:12:29.952522 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance
I20260806 21:12:29.952917 17096 external_mini_cluster.cc:1428] Setting key 6c59e98b9c27bafca70b335db78921a8
I20260806 21:12:29.955040 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.2:0
--local_ip_for_outbound_sockets=127.16.178.2
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:41567
--builtin_ntp_servers=127.16.178.20:43961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 21:12:29.958829 31529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:29.950690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:35331 (local address 127.16.178.62:41567)
0806 21:12:29.950836 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:29.950841 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:29.951945 (+ 1104us) server_negotiation.cc:407] Connection header received
0806 21:12:29.953502 (+ 1557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:29.953506 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:29.953559 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:29.953643 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:29.954768 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:29.955519 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:29.956791 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:29.957023 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:29.957104 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:29.958375 (+ 1271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:29.958378 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:29.958381 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:12:29.958395 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:29.958415 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:12:29.958425 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:29.958483 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:29.958650 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 21:12:29.958709 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 21:12:29.960073 31767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:29.951082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:41567 (local address 127.16.178.1:35331)
0806 21:12:29.951797 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:29.951838 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:29.953229 (+ 1391us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:29.953844 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:29.953853 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:29.953953 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:29.954607 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:29.954624 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:29.955645 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:29.955648 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:29.956651 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:29.956667 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:29.957181 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:29.957185 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:29.957386 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:29.958051 (+ 665us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 21:12:29.958053 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:29.958109 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 21:12:29.958199 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:12:29.958499 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:12:29.958508 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:12:29.958513 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 21:12:29.958644 (+ 131us) client_negotiation.cc:240] Negotiation successful
0806 21:12:29.958961 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":119,"threads_started":1}
I20260806 21:12:29.961817 31764 heartbeater.cc:347] Connected to a master server at 127.16.178.62:41567
I20260806 21:12:29.962121 31764 heartbeater.cc:464] Registering TS with master...
I20260806 21:12:29.962877 31764 heartbeater.cc:511] Master 127.16.178.62:41567 requested a full tablet report, sending...
I20260806 21:12:29.965224 31467 ts_manager.cc:194] Registered new tserver with Master: dc4060b6d9c4403c81bda82dc8f5be38 (127.16.178.1:39927)
I20260806 21:12:29.966878 31467 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.178.1:35331
W20260806 21:12:30.076642 31770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:12:30.076954 31770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:12:30.076994 31770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:12:30.081167 31770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:12:30.081245 31770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:12:30.081300 31770 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.2
I20260806 21:12:30.086419 31770 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:43961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:41567
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31770
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.2
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:12:30.087623 31770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:12:30.089154 31770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:12:30.099462 31789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:30.099490 31783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:30.099452 31785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:30.099740 31770 server_base.cc:1034] running on GCE node
I20260806 21:12:30.100327 31770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:12:30.101024 31770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:12:30.102223 31770 hybrid_clock.cc:648] HybridClock initialized: now 1786050750102203 us; error 40 us; skew 500 ppm
I20260806 21:12:30.104347 31770 webserver.cc:460] Webserver started at http://127.16.178.2:46611/ using document root <none> and password file <none>
I20260806 21:12:30.104979 31770 fs_manager.cc:359] Metadata directory not provided
I20260806 21:12:30.105048 31770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:12:30.105278 31770 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:12:30.107821 31770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/data/instance:
uuid: "a070fd9ca7df4e5cad834208953cb6b8"
format_stamp: "Formatted at 2026-08-06 21:12:30 on dist-test-slave-lgr5"
server_key: "a734be1d513fa310242dfe80ebe9abd3"
server_key_iv: "2a3e3b08d86ddabcaa169df0d534e6c3"
server_key_version: "encryptionkey@0"
I20260806 21:12:30.108501 31770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance:
uuid: "a070fd9ca7df4e5cad834208953cb6b8"
format_stamp: "Formatted at 2026-08-06 21:12:30 on dist-test-slave-lgr5"
server_key: "a734be1d513fa310242dfe80ebe9abd3"
server_key_iv: "2a3e3b08d86ddabcaa169df0d534e6c3"
server_key_version: "encryptionkey@0"
I20260806 21:12:30.113137 31770 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:12:30.116590 31796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:30.118175 31770 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 21:12:30.118337 31770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/wal
uuid: "a070fd9ca7df4e5cad834208953cb6b8"
format_stamp: "Formatted at 2026-08-06 21:12:30 on dist-test-slave-lgr5"
server_key: "a734be1d513fa310242dfe80ebe9abd3"
server_key_iv: "2a3e3b08d86ddabcaa169df0d534e6c3"
server_key_version: "encryptionkey@0"
I20260806 21:12:30.118475 31770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:12:30.144588 31770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:12:30.145363 31770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:12:30.145558 31770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:12:30.146193 31770 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:12:30.147233 31770 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:12:30.147281 31770 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:30.147347 31770 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:12:30.147374 31770 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:30.156759 31770 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.2:46621
I20260806 21:12:30.156792 31963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.2:46621 every 8 connection(s)
I20260806 21:12:30.157783 31770 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
I20260806 21:12:30.163895 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 31770
I20260806 21:12:30.164044 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance
I20260806 21:12:30.164414 17096 external_mini_cluster.cc:1428] Setting key 8d1e94377b15893a0e07d4aac1c381f9
I20260806 21:12:30.166795 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.3:0
--local_ip_for_outbound_sockets=127.16.178.3
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:41567
--builtin_ntp_servers=127.16.178.20:43961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 21:12:30.168284 31529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:30.159768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:57831 (local address 127.16.178.62:41567)
0806 21:12:30.159953 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:30.159958 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:30.161126 (+ 1168us) server_negotiation.cc:407] Connection header received
0806 21:12:30.162597 (+ 1471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:30.162600 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:30.162656 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:30.162742 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:30.163936 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:30.164611 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:30.166122 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:30.166355 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:30.166435 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:30.167799 (+ 1364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:30.167803 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:30.167805 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:12:30.167820 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:30.167841 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:12:30.167849 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:30.167906 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:30.168104 (+ 198us) server_negotiation.cc:299] Negotiation successful
0806 21:12:30.168164 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 21:12:30.169394 31966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:30.160223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:41567 (local address 127.16.178.2:57831)
0806 21:12:30.160972 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:30.161016 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:30.162301 (+ 1285us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:30.162989 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:30.162997 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:30.163086 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:30.163713 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:30.163737 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:30.164758 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:30.164762 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:30.165994 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:30.166006 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:30.166520 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:30.166525 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:30.166747 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:30.167458 (+ 711us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 21:12:30.167460 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:30.167521 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 21:12:30.167616 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:12:30.167930 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:12:30.167939 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:12:30.167947 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 21:12:30.168112 (+ 165us) client_negotiation.cc:240] Negotiation successful
0806 21:12:30.168445 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":122,"threads_started":1}
I20260806 21:12:30.170715 31964 heartbeater.cc:347] Connected to a master server at 127.16.178.62:41567
I20260806 21:12:30.170979 31964 heartbeater.cc:464] Registering TS with master...
I20260806 21:12:30.171561 31964 heartbeater.cc:511] Master 127.16.178.62:41567 requested a full tablet report, sending...
I20260806 21:12:30.172726 31467 ts_manager.cc:194] Registered new tserver with Master: a070fd9ca7df4e5cad834208953cb6b8 (127.16.178.2:46621)
I20260806 21:12:30.173338 31467 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.178.2:57831
W20260806 21:12:30.283106 31969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:12:30.283401 31969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:12:30.283456 31969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:12:30.287485 31969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:12:30.287549 31969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:12:30.287623 31969 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.3
I20260806 21:12:30.292436 31969 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:43961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:41567
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31969
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.3
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:12:30.293562 31969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:12:30.295002 31969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:12:30.305297 31979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:30.305322 31975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:30.305317 31977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:30.305605 31969 server_base.cc:1034] running on GCE node
I20260806 21:12:30.306103 31969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:12:30.306815 31969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:12:30.307973 31969 hybrid_clock.cc:648] HybridClock initialized: now 1786050750307952 us; error 36 us; skew 500 ppm
I20260806 21:12:30.309855 31969 webserver.cc:460] Webserver started at http://127.16.178.3:37355/ using document root <none> and password file <none>
I20260806 21:12:30.310444 31969 fs_manager.cc:359] Metadata directory not provided
I20260806 21:12:30.310513 31969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:12:30.310720 31969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:12:30.312875 31969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/data/instance:
uuid: "3bf4d8483b5b487cb5528c9026950d34"
format_stamp: "Formatted at 2026-08-06 21:12:30 on dist-test-slave-lgr5"
server_key: "410edd23aa135494475b626384a4887f"
server_key_iv: "303e64bdafa10430e29556af57996d84"
server_key_version: "encryptionkey@0"
I20260806 21:12:30.313439 31969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance:
uuid: "3bf4d8483b5b487cb5528c9026950d34"
format_stamp: "Formatted at 2026-08-06 21:12:30 on dist-test-slave-lgr5"
server_key: "410edd23aa135494475b626384a4887f"
server_key_iv: "303e64bdafa10430e29556af57996d84"
server_key_version: "encryptionkey@0"
I20260806 21:12:30.317281 31969 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:12:30.319825 31988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:30.321120 31969 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 21:12:30.321239 31969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/wal
uuid: "3bf4d8483b5b487cb5528c9026950d34"
format_stamp: "Formatted at 2026-08-06 21:12:30 on dist-test-slave-lgr5"
server_key: "410edd23aa135494475b626384a4887f"
server_key_iv: "303e64bdafa10430e29556af57996d84"
server_key_version: "encryptionkey@0"
I20260806 21:12:30.321348 31969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:12:30.336341 31969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:12:30.337128 31969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:12:30.337314 31969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:12:30.337965 31969 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:12:30.339023 31969 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:12:30.339073 31969 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:30.339130 31969 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:12:30.339159 31969 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:30.348201 31969 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.3:44807
I20260806 21:12:30.348228 32150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.3:44807 every 8 connection(s)
I20260806 21:12:30.349341 31969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
I20260806 21:12:30.354815 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 31969
I20260806 21:12:30.354943 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance
I20260806 21:12:30.355278 17096 external_mini_cluster.cc:1428] Setting key 6b24f70980397ebe6d714849ae8ea255
I20260806 21:12:30.359581 31529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:30.351413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:56255 (local address 127.16.178.62:41567)
0806 21:12:30.351589 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:30.351593 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:30.352678 (+ 1085us) server_negotiation.cc:407] Connection header received
0806 21:12:30.354293 (+ 1615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:30.354296 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:30.354358 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:30.354436 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:30.355723 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:30.356352 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:30.357650 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:30.357873 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:30.357973 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:30.359101 (+ 1128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:30.359105 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:30.359107 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:12:30.359119 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:30.359139 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:12:30.359147 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:30.359206 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:30.359357 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 21:12:30.359425 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 21:12:30.360836 32153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:30.351797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:41567 (local address 127.16.178.3:56255)
0806 21:12:30.352521 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:30.352569 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:30.354057 (+ 1488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:30.354658 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:30.354667 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:30.354780 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:30.355518 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:30.355539 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:30.356479 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:30.356483 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:30.357511 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:30.357522 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:30.358032 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:30.358036 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:30.358212 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:30.358833 (+ 621us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 21:12:30.358834 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:30.358878 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 21:12:30.358953 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:12:30.359217 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:12:30.359226 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:12:30.359233 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 21:12:30.359353 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 21:12:30.359688 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":138,"threads_started":1}
I20260806 21:12:30.362152 32151 heartbeater.cc:347] Connected to a master server at 127.16.178.62:41567
I20260806 21:12:30.362426 32151 heartbeater.cc:464] Registering TS with master...
I20260806 21:12:30.363019 32151 heartbeater.cc:511] Master 127.16.178.62:41567 requested a full tablet report, sending...
I20260806 21:12:30.364156 31467 ts_manager.cc:194] Registered new tserver with Master: 3bf4d8483b5b487cb5528c9026950d34 (127.16.178.3:44807)
I20260806 21:12:30.364738 31467 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.178.3:56255
I20260806 21:12:30.369132 17096 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 21:12:30.371876 31529 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:30.371043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33874 (local address 127.16.178.62:41567)
0806 21:12:30.371169 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:30.371173 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:30.371370 (+ 197us) server_negotiation.cc:407] Connection header received
0806 21:12:30.371442 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:30.371444 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:30.371484 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:30.371549 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:30.371745 (+ 196us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33874: BlockingRecv error: recv got EOF from 127.0.0.1:33874 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 21:12:30.372125 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 31567
I20260806 21:12:30.400170 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 31770
I20260806 21:12:30.413932 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 31969
I20260806 21:12:30.435212 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 31420
2026-08-06T21:12:30Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (951 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-06T21:12:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:12:30Z Disabled control of system clock
I20260806 21:12:30.475678 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:45085
--webserver_interface=127.16.178.62
--webserver_port=0
--builtin_ntp_servers=127.16.178.20:36169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:45085
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 21:12:30.592237 32165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:12:30.592545 32165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:12:30.592585 32165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:12:30.596742 32165 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:12:30.596818 32165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:12:30.596832 32165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:12:30.596848 32165 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:12:30.596863 32165 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:12:30.601960 32165 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:36169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:45085
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:45085
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.178.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32165
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:12:30.603152 32165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:12:30.604699 32165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:12:30.613687 32175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:30.613690 32172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:30.613895 32165 server_base.cc:1034] running on GCE node
W20260806 21:12:30.613701 32179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:30.614529 32165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:12:30.615772 32165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:12:30.616942 32165 hybrid_clock.cc:648] HybridClock initialized: now 1786050750616927 us; error 30 us; skew 500 ppm
I20260806 21:12:30.619017 32165 webserver.cc:460] Webserver started at http://127.16.178.62:34083/ using document root <none> and password file <none>
I20260806 21:12:30.619640 32165 fs_manager.cc:359] Metadata directory not provided
I20260806 21:12:30.619714 32165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:12:30.619935 32165 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:12:30.622262 32165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "3cbb0228f3524294993498fabb8a31fa"
format_stamp: "Formatted at 2026-08-06 21:12:30 on dist-test-slave-lgr5"
server_key: "9a4202d27e151fc2b5bb7b394e896078"
server_key_iv: "46539df54e7621d8882a945033d6b082"
server_key_version: "encryptionkey@0"
I20260806 21:12:30.622870 32165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "3cbb0228f3524294993498fabb8a31fa"
format_stamp: "Formatted at 2026-08-06 21:12:30 on dist-test-slave-lgr5"
server_key: "9a4202d27e151fc2b5bb7b394e896078"
server_key_iv: "46539df54e7621d8882a945033d6b082"
server_key_version: "encryptionkey@0"
I20260806 21:12:30.627040 32165 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 21:12:30.630126 32185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:30.631664 32165 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:12:30.631820 32165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "3cbb0228f3524294993498fabb8a31fa"
format_stamp: "Formatted at 2026-08-06 21:12:30 on dist-test-slave-lgr5"
server_key: "9a4202d27e151fc2b5bb7b394e896078"
server_key_iv: "46539df54e7621d8882a945033d6b082"
server_key_version: "encryptionkey@0"
I20260806 21:12:30.631954 32165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:12:30.637288 32165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:12:30.638072 32165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:12:30.638279 32165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:12:30.645896 32165 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.62:45085
I20260806 21:12:30.645915 32267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.62:45085 every 8 connection(s)
I20260806 21:12:30.647047 32165 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:12:30.651180 32268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:12:30.652221 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 32165
I20260806 21:12:30.652344 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:12:30.652666 17096 external_mini_cluster.cc:1428] Setting key b06828f8543f35e89f91511364a34a52
I20260806 21:12:30.658789 32268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cbb0228f3524294993498fabb8a31fa: Bootstrap starting.
I20260806 21:12:30.659183 32272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:30.654384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44254 (local address 127.16.178.62:45085)
0806 21:12:30.655234 (+ 850us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:30.655257 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:30.655299 (+ 42us) server_negotiation.cc:407] Connection header received
0806 21:12:30.656040 (+ 741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:30.656060 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:30.656531 (+ 471us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:30.656883 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:30.657770 (+ 887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:30.657775 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:30.657788 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:12:30.657800 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:30.657863 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:12:30.657880 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:30.658040 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:30.658119 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 21:12:30.658399 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":180,"threads_started":1}
I20260806 21:12:30.661593 32268 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3cbb0228f3524294993498fabb8a31fa: Neither blocks nor log segments found. Creating new log.
I20260806 21:12:30.662432 32268 log.cc:824] T 00000000000000000000000000000000 P 3cbb0228f3524294993498fabb8a31fa: Log is configured to *not* fsync() on all Append() calls
I20260806 21:12:30.664667 32268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cbb0228f3524294993498fabb8a31fa: No bootstrap required, opened a new log
I20260806 21:12:30.668645 32268 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3cbb0228f3524294993498fabb8a31fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cbb0228f3524294993498fabb8a31fa" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 45085 } }
I20260806 21:12:30.668919 32268 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3cbb0228f3524294993498fabb8a31fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:12:30.668973 32268 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3cbb0228f3524294993498fabb8a31fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cbb0228f3524294993498fabb8a31fa, State: Initialized, Role: FOLLOWER
I20260806 21:12:30.669462 32268 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3cbb0228f3524294993498fabb8a31fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cbb0228f3524294993498fabb8a31fa" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 45085 } }
I20260806 21:12:30.669610 32268 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3cbb0228f3524294993498fabb8a31fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:12:30.669675 32268 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3cbb0228f3524294993498fabb8a31fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:12:30.669778 32268 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3cbb0228f3524294993498fabb8a31fa [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:12:30.670796 32268 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3cbb0228f3524294993498fabb8a31fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cbb0228f3524294993498fabb8a31fa" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 45085 } }
I20260806 21:12:30.671159 32268 leader_election.cc:302] T 00000000000000000000000000000000 P 3cbb0228f3524294993498fabb8a31fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3cbb0228f3524294993498fabb8a31fa; no voters:
I20260806 21:12:30.671479 32268 leader_election.cc:288] T 00000000000000000000000000000000 P 3cbb0228f3524294993498fabb8a31fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:12:30.671615 32275 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3cbb0228f3524294993498fabb8a31fa [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:12:30.671837 32275 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3cbb0228f3524294993498fabb8a31fa [term 1 LEADER]: Becoming Leader. State: Replica: 3cbb0228f3524294993498fabb8a31fa, State: Running, Role: LEADER
I20260806 21:12:30.672222 32275 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3cbb0228f3524294993498fabb8a31fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cbb0228f3524294993498fabb8a31fa" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 45085 } }
I20260806 21:12:30.672855 32268 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3cbb0228f3524294993498fabb8a31fa [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:12:30.674118 32276 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cbb0228f3524294993498fabb8a31fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3cbb0228f3524294993498fabb8a31fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cbb0228f3524294993498fabb8a31fa" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 45085 } } }
I20260806 21:12:30.674144 32277 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cbb0228f3524294993498fabb8a31fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3cbb0228f3524294993498fabb8a31fa. Latest consensus state: current_term: 1 leader_uuid: "3cbb0228f3524294993498fabb8a31fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cbb0228f3524294993498fabb8a31fa" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 45085 } } }
I20260806 21:12:30.674353 32276 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cbb0228f3524294993498fabb8a31fa [sys.catalog]: This master's current role is: LEADER
I20260806 21:12:30.674378 32277 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cbb0228f3524294993498fabb8a31fa [sys.catalog]: This master's current role is: LEADER
I20260806 21:12:30.675316 32289 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:12:30.678242 32289 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:12:30.685441 32289 catalog_manager.cc:1347] Generated new cluster ID: 23a9fa28a3b64743abf0b0c0345c9d1b
I20260806 21:12:30.685544 32289 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:12:30.712647 32289 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:12:30.713028 32289 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:12:30.720396 32289 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3cbb0228f3524294993498fabb8a31fa: Generated new TSK 0
I20260806 21:12:30.721225 32289 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:12:30.731295 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.1:0
--local_ip_for_outbound_sockets=127.16.178.1
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:45085
--builtin_ntp_servers=127.16.178.20:36169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 21:12:30.849524 32302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:12:30.849828 32302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:12:30.849885 32302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:12:30.854007 32302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:12:30.854072 32302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:12:30.854151 32302 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.1
I20260806 21:12:30.859127 32302 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:36169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:45085
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.1
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:12:30.860327 32302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:12:30.861764 32302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:12:30.872231 32311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:30.872229 32310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:30.872229 32315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:30.872694 32302 server_base.cc:1034] running on GCE node
I20260806 21:12:30.873165 32302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:12:30.874001 32302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:12:30.875201 32302 hybrid_clock.cc:648] HybridClock initialized: now 1786050750875181 us; error 35 us; skew 500 ppm
I20260806 21:12:30.877308 32302 webserver.cc:460] Webserver started at http://127.16.178.1:39547/ using document root <none> and password file <none>
I20260806 21:12:30.877977 32302 fs_manager.cc:359] Metadata directory not provided
I20260806 21:12:30.878062 32302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:12:30.878304 32302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:12:30.880893 32302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/data/instance:
uuid: "6a114daaed8b4213aa533020b8187bf3"
format_stamp: "Formatted at 2026-08-06 21:12:30 on dist-test-slave-lgr5"
server_key: "38aaa67aa1f49284fe2cebab4c898695"
server_key_iv: "9be13f29a4e1ce9bdd5e1e5f41977452"
server_key_version: "encryptionkey@0"
I20260806 21:12:30.881563 32302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance:
uuid: "6a114daaed8b4213aa533020b8187bf3"
format_stamp: "Formatted at 2026-08-06 21:12:30 on dist-test-slave-lgr5"
server_key: "38aaa67aa1f49284fe2cebab4c898695"
server_key_iv: "9be13f29a4e1ce9bdd5e1e5f41977452"
server_key_version: "encryptionkey@0"
I20260806 21:12:30.885644 32302 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:12:30.888644 32322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:30.890087 32302 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:12:30.890229 32302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/wal
uuid: "6a114daaed8b4213aa533020b8187bf3"
format_stamp: "Formatted at 2026-08-06 21:12:30 on dist-test-slave-lgr5"
server_key: "38aaa67aa1f49284fe2cebab4c898695"
server_key_iv: "9be13f29a4e1ce9bdd5e1e5f41977452"
server_key_version: "encryptionkey@0"
I20260806 21:12:30.890363 32302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:12:30.895617 32302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:12:30.896373 32302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:12:30.896572 32302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:12:30.897212 32302 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:12:30.898268 32302 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:12:30.898329 32302 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:30.898392 32302 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:12:30.898420 32302 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:30.908635 32302 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.1:45559
I20260806 21:12:30.908664 32505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.1:45559 every 8 connection(s)
I20260806 21:12:30.909842 32302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
I20260806 21:12:30.917244 32272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:30.911898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:44337 (local address 127.16.178.62:45085)
0806 21:12:30.912142 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:30.912147 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:30.913441 (+ 1294us) server_negotiation.cc:407] Connection header received
0806 21:12:30.914880 (+ 1439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:30.914883 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:30.914935 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:30.915046 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:30.916735 (+ 1689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:30.916739 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:30.916741 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:12:30.916762 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:30.916795 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:12:30.916806 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:30.916920 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:30.917055 (+ 135us) server_negotiation.cc:299] Negotiation successful
0806 21:12:30.917105 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260806 21:12:30.918390 32510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:30.912381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:45085 (local address 127.16.178.1:44337)
0806 21:12:30.913237 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:30.913298 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:30.914609 (+ 1311us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:30.915345 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:30.915356 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:30.915503 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:30.916336 (+ 833us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 21:12:30.916338 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:30.916398 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 21:12:30.916505 (+ 107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:12:30.916900 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:12:30.916917 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:12:30.916923 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 21:12:30.917097 (+ 174us) client_negotiation.cc:240] Negotiation successful
0806 21:12:30.917435 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":124,"threads_started":1}
I20260806 21:12:30.919229 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 32302
I20260806 21:12:30.919385 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance
I20260806 21:12:30.919750 17096 external_mini_cluster.cc:1428] Setting key 12808c508bdeb8aed406c18166a3acbf
I20260806 21:12:30.920032 32506 heartbeater.cc:347] Connected to a master server at 127.16.178.62:45085
I20260806 21:12:30.920353 32506 heartbeater.cc:464] Registering TS with master...
I20260806 21:12:30.920893 32506 heartbeater.cc:511] Master 127.16.178.62:45085 requested a full tablet report, sending...
I20260806 21:12:30.921983 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.2:0
--local_ip_for_outbound_sockets=127.16.178.2
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:45085
--builtin_ntp_servers=127.16.178.20:36169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 21:12:30.922884 32217 ts_manager.cc:194] Registered new tserver with Master: 6a114daaed8b4213aa533020b8187bf3 (127.16.178.1:45559)
W20260806 21:12:31.044880 32513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:12:31.045187 32513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:12:31.045228 32513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:12:31.049289 32513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:12:31.049371 32513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:12:31.049440 32513 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.2
I20260806 21:12:31.054270 32513 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:36169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:45085
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32513
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.2
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:12:31.055477 32513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:12:31.056885 32513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:12:31.066609 32523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:31.066632 32522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:31.066679 32526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:31.066888 32513 server_base.cc:1034] running on GCE node
I20260806 21:12:31.067339 32513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:12:31.068023 32513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:12:31.069218 32513 hybrid_clock.cc:648] HybridClock initialized: now 1786050751069186 us; error 51 us; skew 500 ppm
I20260806 21:12:31.071206 32513 webserver.cc:460] Webserver started at http://127.16.178.2:44851/ using document root <none> and password file <none>
I20260806 21:12:31.071800 32513 fs_manager.cc:359] Metadata directory not provided
I20260806 21:12:31.071882 32513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:12:31.072121 32513 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:12:31.074373 32513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/data/instance:
uuid: "be797650f2dc4055823a9bb8aa619889"
format_stamp: "Formatted at 2026-08-06 21:12:31 on dist-test-slave-lgr5"
server_key: "011f6aa996bb1ef47111e187eb4511b3"
server_key_iv: "818bc59da9be8480039744bd6555fe33"
server_key_version: "encryptionkey@0"
I20260806 21:12:31.074973 32513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance:
uuid: "be797650f2dc4055823a9bb8aa619889"
format_stamp: "Formatted at 2026-08-06 21:12:31 on dist-test-slave-lgr5"
server_key: "011f6aa996bb1ef47111e187eb4511b3"
server_key_iv: "818bc59da9be8480039744bd6555fe33"
server_key_version: "encryptionkey@0"
I20260806 21:12:31.078720 32513 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260806 21:12:31.081269 32536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:31.082489 32513 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 21:12:31.082613 32513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/wal
uuid: "be797650f2dc4055823a9bb8aa619889"
format_stamp: "Formatted at 2026-08-06 21:12:31 on dist-test-slave-lgr5"
server_key: "011f6aa996bb1ef47111e187eb4511b3"
server_key_iv: "818bc59da9be8480039744bd6555fe33"
server_key_version: "encryptionkey@0"
I20260806 21:12:31.082715 32513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:12:31.087502 32513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:12:31.088143 32513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:12:31.088300 32513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:12:31.088826 32513 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:12:31.089732 32513 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:12:31.089790 32513 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:31.089851 32513 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:12:31.089876 32513 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:31.098475 32513 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.2:34573
I20260806 21:12:31.098501 32737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.2:34573 every 8 connection(s)
I20260806 21:12:31.099383 32513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
I20260806 21:12:31.100328 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 32513
I20260806 21:12:31.100459 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance
I20260806 21:12:31.100737 17096 external_mini_cluster.cc:1428] Setting key 2b354083bc9134de5b3bcbadc16f3b99
I20260806 21:12:31.102876 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.3:0
--local_ip_for_outbound_sockets=127.16.178.3
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:45085
--builtin_ntp_servers=127.16.178.20:36169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 21:12:31.105753 32272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:31.101349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:48117 (local address 127.16.178.62:45085)
0806 21:12:31.101516 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:31.101521 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:31.102723 (+ 1202us) server_negotiation.cc:407] Connection header received
0806 21:12:31.104054 (+ 1331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:31.104057 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:31.104129 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:31.104199 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:31.105317 (+ 1118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:31.105321 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:31.105323 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:12:31.105336 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:31.105358 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:12:31.105366 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:31.105452 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:31.105571 (+ 119us) server_negotiation.cc:299] Negotiation successful
0806 21:12:31.105616 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 21:12:31.106609 32740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:31.101796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:45085 (local address 127.16.178.2:48117)
0806 21:12:31.102568 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:31.102621 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:31.103838 (+ 1217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:31.104386 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:31.104392 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:31.104482 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:31.105060 (+ 578us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 21:12:31.105061 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:31.105102 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 21:12:31.105172 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:12:31.105435 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:12:31.105443 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:12:31.105449 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 21:12:31.105607 (+ 158us) client_negotiation.cc:240] Negotiation successful
0806 21:12:31.105848 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":140,"threads_started":1}
I20260806 21:12:31.107807 32738 heartbeater.cc:347] Connected to a master server at 127.16.178.62:45085
I20260806 21:12:31.108038 32738 heartbeater.cc:464] Registering TS with master...
I20260806 21:12:31.108507 32738 heartbeater.cc:511] Master 127.16.178.62:45085 requested a full tablet report, sending...
I20260806 21:12:31.109586 32217 ts_manager.cc:194] Registered new tserver with Master: be797650f2dc4055823a9bb8aa619889 (127.16.178.2:34573)
W20260806 21:12:31.223481 32741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:12:31.223765 32741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:12:31.223822 32741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:12:31.227865 32741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:12:31.227934 32741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:12:31.228012 32741 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.3
I20260806 21:12:31.232926 32741 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:36169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:45085
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32741
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.3
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:12:31.234050 32741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:12:31.235499 32741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 21:12:31.245349 32741 server_base.cc:1034] running on GCE node
W20260806 21:12:31.246515 32756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:31.246651 32754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:31.246711 32749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:31.247148 32741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:12:31.247812 32741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:12:31.248967 32741 hybrid_clock.cc:648] HybridClock initialized: now 1786050751248933 us; error 46 us; skew 500 ppm
I20260806 21:12:31.250844 32741 webserver.cc:460] Webserver started at http://127.16.178.3:45525/ using document root <none> and password file <none>
I20260806 21:12:31.251441 32741 fs_manager.cc:359] Metadata directory not provided
I20260806 21:12:31.251524 32741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:12:31.251731 32741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:12:31.253983 32741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/data/instance:
uuid: "222d5f6471174e0abd1f6b660b19071b"
format_stamp: "Formatted at 2026-08-06 21:12:31 on dist-test-slave-lgr5"
server_key: "67fefa1ff7593a935826806351acf265"
server_key_iv: "4cc94d4a29cf8561d398aab5b676fc60"
server_key_version: "encryptionkey@0"
I20260806 21:12:31.254624 32741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance:
uuid: "222d5f6471174e0abd1f6b660b19071b"
format_stamp: "Formatted at 2026-08-06 21:12:31 on dist-test-slave-lgr5"
server_key: "67fefa1ff7593a935826806351acf265"
server_key_iv: "4cc94d4a29cf8561d398aab5b676fc60"
server_key_version: "encryptionkey@0"
I20260806 21:12:31.258870 32741 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 21:12:31.261586 32766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:31.262804 32741 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 21:12:31.262919 32741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/wal
uuid: "222d5f6471174e0abd1f6b660b19071b"
format_stamp: "Formatted at 2026-08-06 21:12:31 on dist-test-slave-lgr5"
server_key: "67fefa1ff7593a935826806351acf265"
server_key_iv: "4cc94d4a29cf8561d398aab5b676fc60"
server_key_version: "encryptionkey@0"
I20260806 21:12:31.263027 32741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:12:31.267594 32741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:12:31.268231 32741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:12:31.268393 32741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:12:31.268951 32741 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:12:31.269915 32741 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:12:31.269969 32741 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:31.270025 32741 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:12:31.270053 32741 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:31.279220 32741 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.3:42729
I20260806 21:12:31.279239 480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.3:42729 every 8 connection(s)
I20260806 21:12:31.280248 32741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
I20260806 21:12:31.286382 32272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:31.282101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:37677 (local address 127.16.178.62:45085)
0806 21:12:31.282258 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:31.282263 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:31.283418 (+ 1155us) server_negotiation.cc:407] Connection header received
0806 21:12:31.284748 (+ 1330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:31.284751 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:31.284793 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:31.284868 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:31.286002 (+ 1134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:31.286005 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:31.286007 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:12:31.286018 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:31.286036 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:12:31.286043 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:31.286121 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:31.286212 (+ 91us) server_negotiation.cc:299] Negotiation successful
0806 21:12:31.286255 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 21:12:31.287339 484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:31.282561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:45085 (local address 127.16.178.3:37677)
0806 21:12:31.283259 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:31.283305 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:31.284559 (+ 1254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:31.285038 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:31.285045 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:31.285138 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:31.285727 (+ 589us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 21:12:31.285729 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:31.285786 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 21:12:31.285877 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:12:31.286108 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:12:31.286113 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:12:31.286117 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 21:12:31.286252 (+ 135us) client_negotiation.cc:240] Negotiation successful
0806 21:12:31.286469 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":114,"threads_started":1}
I20260806 21:12:31.288511 481 heartbeater.cc:347] Connected to a master server at 127.16.178.62:45085
I20260806 21:12:31.288754 481 heartbeater.cc:464] Registering TS with master...
I20260806 21:12:31.289206 481 heartbeater.cc:511] Master 127.16.178.62:45085 requested a full tablet report, sending...
I20260806 21:12:31.290259 32217 ts_manager.cc:194] Registered new tserver with Master: 222d5f6471174e0abd1f6b660b19071b (127.16.178.3:42729)
I20260806 21:12:31.290400 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 32741
I20260806 21:12:31.290512 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance
I20260806 21:12:31.290828 17096 external_mini_cluster.cc:1428] Setting key 4dd4d035dd7310b9720caa497b86d84f
I20260806 21:12:31.292932 17096 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 21:12:31.295670 32272 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:12:31.294879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44270 (local address 127.16.178.62:45085)
0806 21:12:31.294995 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:31.294998 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:31.295138 (+ 140us) server_negotiation.cc:407] Connection header received
0806 21:12:31.295233 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:31.295235 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:31.295268 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:31.295334 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:31.295547 (+ 213us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44270: BlockingRecv error: recv got EOF from 127.0.0.1:44270 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 21:12:31.295967 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 32302
I20260806 21:12:31.309638 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 32513
I20260806 21:12:31.323073 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 32741
I20260806 21:12:31.333992 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 32165
2026-08-06T21:12:31Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (893 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:12:31 dist-test-slave-lgr5 krb5kdc[498](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:12:31 dist-test-slave-lgr5 krb5kdc[498](info): set up 2 sockets
Aug 06 21:12:31 dist-test-slave-lgr5 krb5kdc[498](info): commencing operation
krb5kdc: starting...
W20260806 21:12:32.501385 17096 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Aug 06 21:12:32 dist-test-slave-lgr5 krb5kdc[498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:12:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:12:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.178.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
I20260806 21:12:32.636053 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:43275
--webserver_interface=127.16.178.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:44769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:43275
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:12:32.754480 516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:12:32.754837 516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:12:32.754904 516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:12:32.759372 516 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:12:32.759472 516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:12:32.759500 516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:12:32.759522 516 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:12:32.759549 516 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:12:32.765266 516 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:44769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:43275
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:43275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.178.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:12:32.766516 516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:12:32.767637 516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:12:32.777130 529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:32.777130 525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:32.777288 516 server_base.cc:1034] running on GCE node
W20260806 21:12:32.777154 526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:32.777861 516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:12:32.779018 516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:12:32.780164 516 hybrid_clock.cc:648] HybridClock initialized: now 1786050752780142 us; error 32 us; skew 500 ppm
Aug 06 21:12:32 dist-test-slave-lgr5 krb5kdc[498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050752, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:12:32.782927 516 init.cc:377] Logged in from keytab as kudu/127.16.178.62@KRBTEST.COM (short username kudu)
I20260806 21:12:32.783802 516 webserver.cc:460] Webserver started at http://127.16.178.62:33515/ using document root <none> and password file <none>
I20260806 21:12:32.784406 516 fs_manager.cc:359] Metadata directory not provided
I20260806 21:12:32.784471 516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:12:32.784643 516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:12:32.786798 516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "319f57158e304af9af4a3b3143797615"
format_stamp: "Formatted at 2026-08-06 21:12:32 on dist-test-slave-lgr5"
server_key: "c5db58f3a5109cf06c6df3ff129c5228"
server_key_iv: "909904f608d18066cbabfef92a00fd71"
server_key_version: "encryptionkey@0"
I20260806 21:12:32.787331 516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "319f57158e304af9af4a3b3143797615"
format_stamp: "Formatted at 2026-08-06 21:12:32 on dist-test-slave-lgr5"
server_key: "c5db58f3a5109cf06c6df3ff129c5228"
server_key_iv: "909904f608d18066cbabfef92a00fd71"
server_key_version: "encryptionkey@0"
I20260806 21:12:32.791237 516 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260806 21:12:32.793756 540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:32.795101 516 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 21:12:32.795209 516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "319f57158e304af9af4a3b3143797615"
format_stamp: "Formatted at 2026-08-06 21:12:32 on dist-test-slave-lgr5"
server_key: "c5db58f3a5109cf06c6df3ff129c5228"
server_key_iv: "909904f608d18066cbabfef92a00fd71"
server_key_version: "encryptionkey@0"
I20260806 21:12:32.795305 516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:12:32.811192 516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:12:32.814128 516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:12:32.814287 516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:12:32.821522 516 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.62:43275
I20260806 21:12:32.821578 613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.62:43275 every 8 connection(s)
I20260806 21:12:32.822528 516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:12:32.823657 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 516
I20260806 21:12:32.823765 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:12:32.824046 17096 external_mini_cluster.cc:1428] Setting key eff172d98f3ab6da4647d9d538b67802
I20260806 21:12:32.825712 615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:12:32.831676 615 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 319f57158e304af9af4a3b3143797615: Bootstrap starting.
Aug 06 21:12:32 dist-test-slave-lgr5 krb5kdc[498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:12:32.834218 615 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 319f57158e304af9af4a3b3143797615: Neither blocks nor log segments found. Creating new log.
I20260806 21:12:32.835107 615 log.cc:824] T 00000000000000000000000000000000 P 319f57158e304af9af4a3b3143797615: Log is configured to *not* fsync() on all Append() calls
I20260806 21:12:32.837865 615 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 319f57158e304af9af4a3b3143797615: No bootstrap required, opened a new log
I20260806 21:12:32.838655 621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:32.825575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50866 (local address 127.16.178.62:43275)
0806 21:12:32.826041 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:32.826057 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:32.826093 (+ 36us) server_negotiation.cc:407] Connection header received
0806 21:12:32.826761 (+ 668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:32.826778 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:32.827105 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:32.827450 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:32.828296 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:32.829333 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:32.830198 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:32.830703 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:32.830871 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:32.833395 (+ 2524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:32.833401 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:32.833425 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:32.833460 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:32.835216 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:32.835827 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:32.835834 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:32.835841 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:32.836000 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:32.836402 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:32.836404 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:32.836406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:32.836891 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:32.837061 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:32.837366 (+ 305us) server_negotiation.cc:299] Negotiation successful
0806 21:12:32.837746 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":143,"threads_started":1}
I20260806 21:12:32.842041 615 raft_consensus.cc:348] T 00000000000000000000000000000000 P 319f57158e304af9af4a3b3143797615 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "319f57158e304af9af4a3b3143797615" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 43275 } }
I20260806 21:12:32.842304 615 raft_consensus.cc:374] T 00000000000000000000000000000000 P 319f57158e304af9af4a3b3143797615 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:12:32.842370 615 raft_consensus.cc:729] T 00000000000000000000000000000000 P 319f57158e304af9af4a3b3143797615 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 319f57158e304af9af4a3b3143797615, State: Initialized, Role: FOLLOWER
I20260806 21:12:32.842803 615 consensus_queue.cc:260] T 00000000000000000000000000000000 P 319f57158e304af9af4a3b3143797615 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "319f57158e304af9af4a3b3143797615" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 43275 } }
I20260806 21:12:32.842949 615 raft_consensus.cc:388] T 00000000000000000000000000000000 P 319f57158e304af9af4a3b3143797615 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:12:32.843015 615 raft_consensus.cc:482] T 00000000000000000000000000000000 P 319f57158e304af9af4a3b3143797615 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:12:32.843119 615 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 319f57158e304af9af4a3b3143797615 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:12:32.844156 615 raft_consensus.cc:504] T 00000000000000000000000000000000 P 319f57158e304af9af4a3b3143797615 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "319f57158e304af9af4a3b3143797615" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 43275 } }
I20260806 21:12:32.844497 615 leader_election.cc:302] T 00000000000000000000000000000000 P 319f57158e304af9af4a3b3143797615 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 319f57158e304af9af4a3b3143797615; no voters:
I20260806 21:12:32.844825 615 leader_election.cc:288] T 00000000000000000000000000000000 P 319f57158e304af9af4a3b3143797615 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:12:32.844944 623 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 319f57158e304af9af4a3b3143797615 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:12:32.845181 623 raft_consensus.cc:686] T 00000000000000000000000000000000 P 319f57158e304af9af4a3b3143797615 [term 1 LEADER]: Becoming Leader. State: Replica: 319f57158e304af9af4a3b3143797615, State: Running, Role: LEADER
I20260806 21:12:32.845587 623 consensus_queue.cc:237] T 00000000000000000000000000000000 P 319f57158e304af9af4a3b3143797615 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "319f57158e304af9af4a3b3143797615" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 43275 } }
I20260806 21:12:32.846197 615 sys_catalog.cc:564] T 00000000000000000000000000000000 P 319f57158e304af9af4a3b3143797615 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:12:32.847465 624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 319f57158e304af9af4a3b3143797615 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "319f57158e304af9af4a3b3143797615" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "319f57158e304af9af4a3b3143797615" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 43275 } } }
I20260806 21:12:32.847468 625 sys_catalog.cc:455] T 00000000000000000000000000000000 P 319f57158e304af9af4a3b3143797615 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 319f57158e304af9af4a3b3143797615. Latest consensus state: current_term: 1 leader_uuid: "319f57158e304af9af4a3b3143797615" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "319f57158e304af9af4a3b3143797615" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 43275 } } }
I20260806 21:12:32.847688 625 sys_catalog.cc:458] T 00000000000000000000000000000000 P 319f57158e304af9af4a3b3143797615 [sys.catalog]: This master's current role is: LEADER
I20260806 21:12:32.847983 624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 319f57158e304af9af4a3b3143797615 [sys.catalog]: This master's current role is: LEADER
I20260806 21:12:32.848114 633 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:12:32.851998 633 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:12:32.859251 633 catalog_manager.cc:1347] Generated new cluster ID: 2399f65ae4be4644b92ac7ee936a921a
I20260806 21:12:32.859356 633 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:12:32.875758 633 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:12:32.876852 633 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:12:32.881944 633 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 319f57158e304af9af4a3b3143797615: Generated new TSK 0
I20260806 21:12:32.882833 633 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.178.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab.
Entry for principal kudu/127.16.178.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab.
I20260806 21:12:32.918984 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.1:0
--local_ip_for_outbound_sockets=127.16.178.1
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:43275
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab
--principal=kudu/127.16.178.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:44769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:12:33.034245 652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:12:33.034536 652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:12:33.034579 652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:12:33.038771 652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:12:33.038885 652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:12:33.038949 652 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.1
I20260806 21:12:33.044474 652 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:44769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab
--principal=kudu/127.16.178.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:43275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.652
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.1
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:12:33.045646 652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:12:33.046762 652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:12:33.056420 660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:33.056420 657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:33.056473 652 server_base.cc:1034] running on GCE node
W20260806 21:12:33.056517 658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:33.057121 652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:12:33.057806 652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:12:33.058983 652 hybrid_clock.cc:648] HybridClock initialized: now 1786050753058963 us; error 54 us; skew 500 ppm
Aug 06 21:12:33 dist-test-slave-lgr5 krb5kdc[498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050753, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:12:33.062186 652 init.cc:377] Logged in from keytab as kudu/127.16.178.1@KRBTEST.COM (short username kudu)
I20260806 21:12:33.063212 652 webserver.cc:460] Webserver started at http://127.16.178.1:41133/ using document root <none> and password file <none>
I20260806 21:12:33.063788 652 fs_manager.cc:359] Metadata directory not provided
I20260806 21:12:33.063853 652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:12:33.064035 652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:12:33.066238 652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/data/instance:
uuid: "25a778364d7d42639e4b4224d5aafed8"
format_stamp: "Formatted at 2026-08-06 21:12:33 on dist-test-slave-lgr5"
server_key: "90ab13ad1917670b0eb601dd182214bb"
server_key_iv: "99810b961fd037a41e2e1e330bbacc05"
server_key_version: "encryptionkey@0"
I20260806 21:12:33.066766 652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance:
uuid: "25a778364d7d42639e4b4224d5aafed8"
format_stamp: "Formatted at 2026-08-06 21:12:33 on dist-test-slave-lgr5"
server_key: "90ab13ad1917670b0eb601dd182214bb"
server_key_iv: "99810b961fd037a41e2e1e330bbacc05"
server_key_version: "encryptionkey@0"
I20260806 21:12:33.070470 652 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 21:12:33.072853 670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:33.074074 652 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 21:12:33.074194 652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/wal
uuid: "25a778364d7d42639e4b4224d5aafed8"
format_stamp: "Formatted at 2026-08-06 21:12:33 on dist-test-slave-lgr5"
server_key: "90ab13ad1917670b0eb601dd182214bb"
server_key_iv: "99810b961fd037a41e2e1e330bbacc05"
server_key_version: "encryptionkey@0"
I20260806 21:12:33.074290 652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:12:33.088766 652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:12:33.091826 652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:12:33.092005 652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:12:33.092609 652 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:12:33.093434 652 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:12:33.093482 652 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:33.093537 652 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:12:33.093567 652 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:33.101891 652 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.1:41463
I20260806 21:12:33.101959 841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.1:41463 every 8 connection(s)
I20260806 21:12:33.102792 652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
I20260806 21:12:33.106477 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 652
I20260806 21:12:33.106587 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance
I20260806 21:12:33.106856 17096 external_mini_cluster.cc:1428] Setting key ba813987333d4d21249c2bf732083e91
Aug 06 21:12:33 dist-test-slave-lgr5 krb5kdc[498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050753, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:12:33.117416 621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:33.104593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:52749 (local address 127.16.178.62:43275)
0806 21:12:33.104736 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:33.104740 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:33.105549 (+ 809us) server_negotiation.cc:407] Connection header received
0806 21:12:33.107017 (+ 1468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:33.107021 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:33.107080 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:33.107169 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:33.108661 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:33.109373 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:33.110659 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:33.110885 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:33.110965 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:33.113701 (+ 2736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:33.113705 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:33.113708 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:33.113736 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:33.115136 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:33.115775 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:33.115777 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:33.115778 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:33.115849 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:33.116260 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:33.116263 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:33.116264 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:33.116477 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:33.116566 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:33.117186 (+ 620us) server_negotiation.cc:299] Negotiation successful
0806 21:12:33.117291 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
WARNING: no policy specified for kudu/127.16.178.2@KRBTEST.COM; defaulting to no policy
I20260806 21:12:33.118432 848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:33.104961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:43275 (local address 127.16.178.1:52749)
0806 21:12:33.105404 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:33.105446 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:33.106805 (+ 1359us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:33.107369 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:33.107379 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:33.107836 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:33.108523 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:33.108540 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:33.109510 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:33.109513 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:33.110524 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:33.110537 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:33.111044 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:33.111048 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:33.111242 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:33.111818 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:33.111839 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:33.113530 (+ 1691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:12:33.115262 (+ 1732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:33.115268 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:33.115285 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:33.115678 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:33.115926 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:33.115929 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:33.115931 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:33.116159 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:33.116592 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:12:33.116599 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:12:33.116933 (+ 334us) client_negotiation.cc:769] Sending connection context
0806 21:12:33.117157 (+ 224us) client_negotiation.cc:240] Negotiation successful
0806 21:12:33.117497 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":113,"threads_started":1}
I20260806 21:12:33.119696 844 heartbeater.cc:347] Connected to a master server at 127.16.178.62:43275
I20260806 21:12:33.119974 844 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.2@KRBTEST.COM" created.
I20260806 21:12:33.120584 844 heartbeater.cc:511] Master 127.16.178.62:43275 requested a full tablet report, sending...
I20260806 21:12:33.122255 563 ts_manager.cc:194] Registered new tserver with Master: 25a778364d7d42639e4b4224d5aafed8 (127.16.178.1:41463)
I20260806 21:12:33.123464 563 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.1@KRBTEST.COM'} at 127.16.178.1:52749
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab.
Entry for principal kudu/127.16.178.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab.
I20260806 21:12:33.135432 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.2:0
--local_ip_for_outbound_sockets=127.16.178.2
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:43275
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab
--principal=kudu/127.16.178.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:44769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:12:33.253515 852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:12:33.253788 852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:12:33.253837 852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:12:33.258029 852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:12:33.258098 852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:12:33.258157 852 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.2
I20260806 21:12:33.263782 852 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:44769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab
--principal=kudu/127.16.178.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:43275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.852
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.2
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:12:33.265000 852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:12:33.266042 852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:12:33.279018 861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:33.279245 868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:33.279049 859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:33.279368 852 server_base.cc:1034] running on GCE node
I20260806 21:12:33.279973 852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:12:33.282331 852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:12:33.284360 852 hybrid_clock.cc:648] HybridClock initialized: now 1786050753284264 us; error 124 us; skew 500 ppm
Aug 06 21:12:33 dist-test-slave-lgr5 krb5kdc[498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050753, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:12:33.288589 852 init.cc:377] Logged in from keytab as kudu/127.16.178.2@KRBTEST.COM (short username kudu)
I20260806 21:12:33.289623 852 webserver.cc:460] Webserver started at http://127.16.178.2:42961/ using document root <none> and password file <none>
I20260806 21:12:33.290235 852 fs_manager.cc:359] Metadata directory not provided
I20260806 21:12:33.290343 852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:12:33.290629 852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:12:33.294075 852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/data/instance:
uuid: "39d51b25a89d4580b56046bd767e9211"
format_stamp: "Formatted at 2026-08-06 21:12:33 on dist-test-slave-lgr5"
server_key: "1ba7e5f2deb2375d954fb3ec4459416b"
server_key_iv: "f6f05a7e82584ea410530b6f1515885b"
server_key_version: "encryptionkey@0"
I20260806 21:12:33.295140 852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance:
uuid: "39d51b25a89d4580b56046bd767e9211"
format_stamp: "Formatted at 2026-08-06 21:12:33 on dist-test-slave-lgr5"
server_key: "1ba7e5f2deb2375d954fb3ec4459416b"
server_key_iv: "f6f05a7e82584ea410530b6f1515885b"
server_key_version: "encryptionkey@0"
I20260806 21:12:33.300704 852 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 21:12:33.307973 879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:33.309935 852 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.003s sys 0.003s
I20260806 21:12:33.310110 852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/wal
uuid: "39d51b25a89d4580b56046bd767e9211"
format_stamp: "Formatted at 2026-08-06 21:12:33 on dist-test-slave-lgr5"
server_key: "1ba7e5f2deb2375d954fb3ec4459416b"
server_key_iv: "f6f05a7e82584ea410530b6f1515885b"
server_key_version: "encryptionkey@0"
I20260806 21:12:33.310282 852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:12:33.323526 852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:12:33.326866 852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:12:33.327085 852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:12:33.327754 852 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:12:33.328944 852 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:12:33.329010 852 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:33.329068 852 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:12:33.329097 852 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:33.339867 852 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.2:38433
I20260806 21:12:33.339893 1053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.2:38433 every 8 connection(s)
I20260806 21:12:33.340924 852 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
I20260806 21:12:33.348318 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 852
I20260806 21:12:33.348444 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance
I20260806 21:12:33.348737 17096 external_mini_cluster.cc:1428] Setting key 318dcfd8f4981d77bf6599c66e736b41
Aug 06 21:12:33 dist-test-slave-lgr5 krb5kdc[498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050753, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.2@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:12:33.356324 621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:33.342826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:56371 (local address 127.16.178.62:43275)
0806 21:12:33.342999 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:33.343004 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:33.343883 (+ 879us) server_negotiation.cc:407] Connection header received
0806 21:12:33.345434 (+ 1551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:33.345438 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:33.345493 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:33.345570 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:33.346873 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:33.347477 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:33.348888 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:33.349201 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:33.349298 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:33.352315 (+ 3017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:33.352320 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:33.352324 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:33.352360 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:33.353837 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:33.354428 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:33.354432 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:33.354433 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:33.354533 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:33.354953 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:33.354957 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:33.354959 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:33.355224 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:33.355335 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:33.355991 (+ 656us) server_negotiation.cc:299] Negotiation successful
0806 21:12:33.356165 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 21:12:33.357430 1060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:33.343235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:43275 (local address 127.16.178.2:56371)
0806 21:12:33.343725 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:33.343772 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:33.345101 (+ 1329us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:33.345742 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:33.345749 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:33.346119 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:33.346700 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:33.346716 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:33.347623 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:33.347626 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:33.348755 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:33.348768 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:33.349386 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:33.349391 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:33.349590 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:33.350302 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:33.350325 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:33.352030 (+ 1705us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:12:33.353968 (+ 1938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:33.353974 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:33.353992 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:33.354301 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:33.354657 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:33.354660 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:33.354661 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:33.354843 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:33.355339 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:12:33.355346 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:12:33.355669 (+ 323us) client_negotiation.cc:769] Sending connection context
0806 21:12:33.355884 (+ 215us) client_negotiation.cc:240] Negotiation successful
0806 21:12:33.356246 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":125,"threads_started":1}
I20260806 21:12:33.358716 1055 heartbeater.cc:347] Connected to a master server at 127.16.178.62:43275
I20260806 21:12:33.358960 1055 heartbeater.cc:464] Registering TS with master...
I20260806 21:12:33.359488 1055 heartbeater.cc:511] Master 127.16.178.62:43275 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.16.178.3@KRBTEST.COM; defaulting to no policy
I20260806 21:12:33.360661 563 ts_manager.cc:194] Registered new tserver with Master: 39d51b25a89d4580b56046bd767e9211 (127.16.178.2:38433)
I20260806 21:12:33.361299 563 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.2@KRBTEST.COM'} at 127.16.178.2:56371
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab.
Entry for principal kudu/127.16.178.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab.
I20260806 21:12:33.379470 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.3:0
--local_ip_for_outbound_sockets=127.16.178.3
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:43275
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab
--principal=kudu/127.16.178.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:44769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:12:33.497416 1063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:12:33.497701 1063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:12:33.497745 1063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:12:33.501811 1063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:12:33.501871 1063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:12:33.501935 1063 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.3
I20260806 21:12:33.507266 1063 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:44769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab
--principal=kudu/127.16.178.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:43275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1063
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.3
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:12:33.508378 1063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:12:33.509387 1063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:12:33.519496 1069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:33.519557 1063 server_base.cc:1034] running on GCE node
W20260806 21:12:33.519505 1075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:12:33.519500 1072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:12:33.520185 1063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:12:33.520889 1063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:12:33.522058 1063 hybrid_clock.cc:648] HybridClock initialized: now 1786050753522029 us; error 44 us; skew 500 ppm
Aug 06 21:12:33 dist-test-slave-lgr5 krb5kdc[498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050753, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:12:33.524762 1063 init.cc:377] Logged in from keytab as kudu/127.16.178.3@KRBTEST.COM (short username kudu)
I20260806 21:12:33.525636 1063 webserver.cc:460] Webserver started at http://127.16.178.3:41717/ using document root <none> and password file <none>
I20260806 21:12:33.526175 1063 fs_manager.cc:359] Metadata directory not provided
I20260806 21:12:33.526242 1063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:12:33.526471 1063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:12:33.528574 1063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/data/instance:
uuid: "0ca8e269e476459ebdf315facaaa0ab5"
format_stamp: "Formatted at 2026-08-06 21:12:33 on dist-test-slave-lgr5"
server_key: "29728d246e1549d94e0b50054b2a9cd9"
server_key_iv: "4564a3289b248aa098586824091bcfc3"
server_key_version: "encryptionkey@0"
I20260806 21:12:33.529119 1063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance:
uuid: "0ca8e269e476459ebdf315facaaa0ab5"
format_stamp: "Formatted at 2026-08-06 21:12:33 on dist-test-slave-lgr5"
server_key: "29728d246e1549d94e0b50054b2a9cd9"
server_key_iv: "4564a3289b248aa098586824091bcfc3"
server_key_version: "encryptionkey@0"
I20260806 21:12:33.532728 1063 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 21:12:33.535065 1085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:33.536218 1063 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 21:12:33.536324 1063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/wal
uuid: "0ca8e269e476459ebdf315facaaa0ab5"
format_stamp: "Formatted at 2026-08-06 21:12:33 on dist-test-slave-lgr5"
server_key: "29728d246e1549d94e0b50054b2a9cd9"
server_key_iv: "4564a3289b248aa098586824091bcfc3"
server_key_version: "encryptionkey@0"
I20260806 21:12:33.536441 1063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:12:33.549786 1063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:12:33.552893 1063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:12:33.553076 1063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:12:33.553668 1063 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:12:33.554487 1063 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:12:33.554535 1063 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:33.554589 1063 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:12:33.554613 1063 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:12:33.563172 1063 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.3:35635
I20260806 21:12:33.563220 1280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.3:35635 every 8 connection(s)
I20260806 21:12:33.564110 1063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
I20260806 21:12:33.567513 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 1063
I20260806 21:12:33.567639 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance
I20260806 21:12:33.567912 17096 external_mini_cluster.cc:1428] Setting key 0358a70e443f63f364217a2f6100b6f3
Aug 06 21:12:33 dist-test-slave-lgr5 krb5kdc[498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050753, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.3@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:12:33.577453 621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:33.565927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:51901 (local address 127.16.178.62:43275)
0806 21:12:33.566062 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:33.566066 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:33.566932 (+ 866us) server_negotiation.cc:407] Connection header received
0806 21:12:33.568439 (+ 1507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:33.568443 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:33.568499 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:33.568591 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:33.570069 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:33.570601 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:33.571668 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:33.571868 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:33.571938 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:33.574339 (+ 2401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:33.574342 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:33.574344 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:33.574391 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:33.575694 (+ 1303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:33.576161 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:33.576164 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:33.576165 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:33.576237 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:33.576501 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:33.576503 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:33.576504 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:33.576672 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:33.576748 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:33.577190 (+ 442us) server_negotiation.cc:299] Negotiation successful
0806 21:12:33.577327 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 21:12:33.578258 1283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:33.566251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:43275 (local address 127.16.178.3:51901)
0806 21:12:33.566741 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:33.566799 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:33.568207 (+ 1408us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:33.568782 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:33.568791 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:33.569250 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:12:33.569934 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:33.569955 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:33.570703 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:33.570706 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:33.571576 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:33.571586 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:33.571974 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:33.571976 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:33.572164 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:33.572730 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:12:33.572747 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:12:33.574143 (+ 1396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:12:33.575793 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:33.575797 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:33.575808 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:33.576048 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:33.576302 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:12:33.576304 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:12:33.576305 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:12:33.576430 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:12:33.576746 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:12:33.576751 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:12:33.576971 (+ 220us) client_negotiation.cc:769] Sending connection context
0806 21:12:33.577119 (+ 148us) client_negotiation.cc:240] Negotiation successful
0806 21:12:33.577387 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":120,"threads_started":1}
I20260806 21:12:33.579322 1281 heartbeater.cc:347] Connected to a master server at 127.16.178.62:43275
I20260806 21:12:33.579550 1281 heartbeater.cc:464] Registering TS with master...
I20260806 21:12:33.580008 1281 heartbeater.cc:511] Master 127.16.178.62:43275 requested a full tablet report, sending...
I20260806 21:12:33.581025 563 ts_manager.cc:194] Registered new tserver with Master: 0ca8e269e476459ebdf315facaaa0ab5 (127.16.178.3:35635)
I20260806 21:12:33.581691 563 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.3@KRBTEST.COM'} at 127.16.178.3:51901
I20260806 21:12:33.581826 17096 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 21:12:33.590467 621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:33.583949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50872 (local address 127.16.178.62:43275)
0806 21:12:33.584070 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:33.584074 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:33.584299 (+ 225us) server_negotiation.cc:407] Connection header received
0806 21:12:33.584404 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:33.584407 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:33.584449 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:33.584508 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:12:33.585136 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:33.585622 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:33.586375 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:33.586572 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:33.586635 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:33.587787 (+ 1152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:12:33.587791 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:12:33.587792 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:12:33.587813 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:12:33.588953 (+ 1140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:33.589334 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:33.589337 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:33.589339 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:33.589412 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:12:33.589662 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:12:33.589665 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:12:33.589667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:12:33.589871 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:12:33.589988 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:33.590218 (+ 230us) server_negotiation.cc:299] Negotiation successful
0806 21:12:33.590323 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 21:12:33.593031 563 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50872:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:12:33.595975 563 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:12:33.610352 1299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:33.604579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:41463 (local address 127.0.0.1:50002)
0806 21:12:33.605771 (+ 1192us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:33.605784 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:33.605883 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:33.606319 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:33.606325 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:33.606351 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:12:33.606555 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:33.606563 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:33.607759 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:33.607762 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:33.609018 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:33.609029 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:33.609950 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:33.609953 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:33.610032 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:33.610062 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 21:12:33.610133 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 21:12:33.610216 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1129,"mutex_wait_us":56,"thread_start_us":110,"threads_started":1}
I20260806 21:12:33.610417 1301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:33.605178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50002 (local address 127.16.178.1:41463)
0806 21:12:33.605531 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:33.605550 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:33.605865 (+ 315us) server_negotiation.cc:407] Connection header received
0806 21:12:33.605983 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:33.605987 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:33.606152 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:33.606287 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:12:33.606677 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:33.607633 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:33.609137 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:33.609831 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:33.609938 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:33.610105 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:33.610163 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 21:12:33.610254 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":73,"threads_started":1}
I20260806 21:12:33.611181 1298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:33.604481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.2:38433 (local address 127.0.0.1:50326)
0806 21:12:33.605349 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:33.605401 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:33.605639 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:33.606393 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:33.606396 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:33.606415 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:12:33.606592 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:33.606598 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:33.608119 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:33.608123 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:33.609377 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:33.609384 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:33.610382 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:33.610385 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:33.610895 (+ 510us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:33.610924 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 21:12:33.610986 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 21:12:33.611054 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":690,"mutex_wait_us":96,"thread_start_us":69,"threads_started":1}
I20260806 21:12:33.611275 1300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:33.604639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50326 (local address 127.16.178.2:38433)
0806 21:12:33.605004 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:33.605027 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:33.605499 (+ 472us) server_negotiation.cc:407] Connection header received
0806 21:12:33.605830 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:33.605837 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:33.606046 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:33.606173 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:12:33.606760 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:33.607941 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:33.609495 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:33.610257 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:33.610365 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:33.610501 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:33.611043 (+ 542us) server_negotiation.cc:299] Negotiation successful
0806 21:12:33.611134 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":160,"threads_started":1}
I20260806 21:12:33.612031 1303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:33.605194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.3:35635 (local address 127.0.0.1:45944)
0806 21:12:33.606014 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:33.606026 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:33.606090 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:33.606768 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:33.606771 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:33.606790 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:12:33.606867 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:33.606878 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:33.608211 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:33.608214 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:33.609366 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:33.609378 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:33.611661 (+ 2283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:33.611665 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:33.611748 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:33.611772 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 21:12:33.611837 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 21:12:33.611906 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":760,"mutex_wait_us":5,"thread_start_us":89,"threads_started":1}
I20260806 21:12:33.612576 1304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:33.605601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45944 (local address 127.16.178.3:35635)
0806 21:12:33.606285 (+ 684us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:33.606302 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:33.606322 (+ 20us) server_negotiation.cc:407] Connection header received
0806 21:12:33.606387 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:33.606392 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:33.606567 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:33.606696 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:12:33.606984 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:33.608059 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:33.610579 (+ 2520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:33.611345 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:33.611446 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:33.611593 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:33.612339 (+ 746us) server_negotiation.cc:299] Negotiation successful
0806 21:12:33.612419 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":131,"threads_started":1}
I20260806 21:12:33.613786 746 tablet_service.cc:1467] Processing CreateTablet for tablet e474ca118b1941e69ad9fb4964dcf8c5 (DEFAULT_TABLE table=test-table [id=1e2b5a69183f48fdb7e737ea0aef2449]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:12:33.614216 947 tablet_service.cc:1467] Processing CreateTablet for tablet e474ca118b1941e69ad9fb4964dcf8c5 (DEFAULT_TABLE table=test-table [id=1e2b5a69183f48fdb7e737ea0aef2449]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:12:33.615011 746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e474ca118b1941e69ad9fb4964dcf8c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:12:33.615118 947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e474ca118b1941e69ad9fb4964dcf8c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:12:33.615155 1174 tablet_service.cc:1467] Processing CreateTablet for tablet e474ca118b1941e69ad9fb4964dcf8c5 (DEFAULT_TABLE table=test-table [id=1e2b5a69183f48fdb7e737ea0aef2449]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:12:33.616189 1174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e474ca118b1941e69ad9fb4964dcf8c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:12:33.622340 1305 tablet_bootstrap.cc:492] T e474ca118b1941e69ad9fb4964dcf8c5 P 39d51b25a89d4580b56046bd767e9211: Bootstrap starting.
I20260806 21:12:33.623204 1309 tablet_bootstrap.cc:492] T e474ca118b1941e69ad9fb4964dcf8c5 P 0ca8e269e476459ebdf315facaaa0ab5: Bootstrap starting.
I20260806 21:12:33.624450 1308 tablet_bootstrap.cc:492] T e474ca118b1941e69ad9fb4964dcf8c5 P 25a778364d7d42639e4b4224d5aafed8: Bootstrap starting.
I20260806 21:12:33.625056 1305 tablet_bootstrap.cc:654] T e474ca118b1941e69ad9fb4964dcf8c5 P 39d51b25a89d4580b56046bd767e9211: Neither blocks nor log segments found. Creating new log.
I20260806 21:12:33.625176 1309 tablet_bootstrap.cc:654] T e474ca118b1941e69ad9fb4964dcf8c5 P 0ca8e269e476459ebdf315facaaa0ab5: Neither blocks nor log segments found. Creating new log.
I20260806 21:12:33.626061 1305 log.cc:824] T e474ca118b1941e69ad9fb4964dcf8c5 P 39d51b25a89d4580b56046bd767e9211: Log is configured to *not* fsync() on all Append() calls
I20260806 21:12:33.626456 1309 log.cc:824] T e474ca118b1941e69ad9fb4964dcf8c5 P 0ca8e269e476459ebdf315facaaa0ab5: Log is configured to *not* fsync() on all Append() calls
I20260806 21:12:33.626679 1308 tablet_bootstrap.cc:654] T e474ca118b1941e69ad9fb4964dcf8c5 P 25a778364d7d42639e4b4224d5aafed8: Neither blocks nor log segments found. Creating new log.
I20260806 21:12:33.627496 1308 log.cc:824] T e474ca118b1941e69ad9fb4964dcf8c5 P 25a778364d7d42639e4b4224d5aafed8: Log is configured to *not* fsync() on all Append() calls
I20260806 21:12:33.628710 1305 tablet_bootstrap.cc:492] T e474ca118b1941e69ad9fb4964dcf8c5 P 39d51b25a89d4580b56046bd767e9211: No bootstrap required, opened a new log
I20260806 21:12:33.628734 1309 tablet_bootstrap.cc:492] T e474ca118b1941e69ad9fb4964dcf8c5 P 0ca8e269e476459ebdf315facaaa0ab5: No bootstrap required, opened a new log
I20260806 21:12:33.628918 1305 ts_tablet_manager.cc:1397] T e474ca118b1941e69ad9fb4964dcf8c5 P 39d51b25a89d4580b56046bd767e9211: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 21:12:33.628925 1309 ts_tablet_manager.cc:1397] T e474ca118b1941e69ad9fb4964dcf8c5 P 0ca8e269e476459ebdf315facaaa0ab5: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 21:12:33.631995 1309 raft_consensus.cc:348] T e474ca118b1941e69ad9fb4964dcf8c5 P 0ca8e269e476459ebdf315facaaa0ab5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d51b25a89d4580b56046bd767e9211" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38433 } } peers { permanent_uuid: "25a778364d7d42639e4b4224d5aafed8" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 41463 } } peers { permanent_uuid: "0ca8e269e476459ebdf315facaaa0ab5" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 35635 } }
I20260806 21:12:33.631996 1305 raft_consensus.cc:348] T e474ca118b1941e69ad9fb4964dcf8c5 P 39d51b25a89d4580b56046bd767e9211 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d51b25a89d4580b56046bd767e9211" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38433 } } peers { permanent_uuid: "25a778364d7d42639e4b4224d5aafed8" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 41463 } } peers { permanent_uuid: "0ca8e269e476459ebdf315facaaa0ab5" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 35635 } }
I20260806 21:12:33.632354 1305 raft_consensus.cc:374] T e474ca118b1941e69ad9fb4964dcf8c5 P 39d51b25a89d4580b56046bd767e9211 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:12:33.632354 1309 raft_consensus.cc:374] T e474ca118b1941e69ad9fb4964dcf8c5 P 0ca8e269e476459ebdf315facaaa0ab5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:12:33.632421 1309 raft_consensus.cc:729] T e474ca118b1941e69ad9fb4964dcf8c5 P 0ca8e269e476459ebdf315facaaa0ab5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ca8e269e476459ebdf315facaaa0ab5, State: Initialized, Role: FOLLOWER
I20260806 21:12:33.632434 1305 raft_consensus.cc:729] T e474ca118b1941e69ad9fb4964dcf8c5 P 39d51b25a89d4580b56046bd767e9211 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39d51b25a89d4580b56046bd767e9211, State: Initialized, Role: FOLLOWER
I20260806 21:12:33.632943 1305 consensus_queue.cc:260] T e474ca118b1941e69ad9fb4964dcf8c5 P 39d51b25a89d4580b56046bd767e9211 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d51b25a89d4580b56046bd767e9211" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38433 } } peers { permanent_uuid: "25a778364d7d42639e4b4224d5aafed8" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 41463 } } peers { permanent_uuid: "0ca8e269e476459ebdf315facaaa0ab5" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 35635 } }
I20260806 21:12:33.632946 1309 consensus_queue.cc:260] T e474ca118b1941e69ad9fb4964dcf8c5 P 0ca8e269e476459ebdf315facaaa0ab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d51b25a89d4580b56046bd767e9211" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38433 } } peers { permanent_uuid: "25a778364d7d42639e4b4224d5aafed8" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 41463 } } peers { permanent_uuid: "0ca8e269e476459ebdf315facaaa0ab5" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 35635 } }
I20260806 21:12:33.633742 1055 heartbeater.cc:503] Master 127.16.178.62:43275 was elected leader, sending a full tablet report...
I20260806 21:12:33.634021 1281 heartbeater.cc:503] Master 127.16.178.62:43275 was elected leader, sending a full tablet report...
I20260806 21:12:33.634164 1305 ts_tablet_manager.cc:1428] T e474ca118b1941e69ad9fb4964dcf8c5 P 39d51b25a89d4580b56046bd767e9211: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 21:12:33.634543 1308 tablet_bootstrap.cc:492] T e474ca118b1941e69ad9fb4964dcf8c5 P 25a778364d7d42639e4b4224d5aafed8: No bootstrap required, opened a new log
I20260806 21:12:33.634768 1308 ts_tablet_manager.cc:1397] T e474ca118b1941e69ad9fb4964dcf8c5 P 25a778364d7d42639e4b4224d5aafed8: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20260806 21:12:33.635351 1309 ts_tablet_manager.cc:1428] T e474ca118b1941e69ad9fb4964dcf8c5 P 0ca8e269e476459ebdf315facaaa0ab5: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 21:12:33.639082 1308 raft_consensus.cc:348] T e474ca118b1941e69ad9fb4964dcf8c5 P 25a778364d7d42639e4b4224d5aafed8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d51b25a89d4580b56046bd767e9211" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38433 } } peers { permanent_uuid: "25a778364d7d42639e4b4224d5aafed8" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 41463 } } peers { permanent_uuid: "0ca8e269e476459ebdf315facaaa0ab5" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 35635 } }
I20260806 21:12:33.639400 1308 raft_consensus.cc:374] T e474ca118b1941e69ad9fb4964dcf8c5 P 25a778364d7d42639e4b4224d5aafed8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:12:33.639473 1308 raft_consensus.cc:729] T e474ca118b1941e69ad9fb4964dcf8c5 P 25a778364d7d42639e4b4224d5aafed8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25a778364d7d42639e4b4224d5aafed8, State: Initialized, Role: FOLLOWER
I20260806 21:12:33.640005 1308 consensus_queue.cc:260] T e474ca118b1941e69ad9fb4964dcf8c5 P 25a778364d7d42639e4b4224d5aafed8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d51b25a89d4580b56046bd767e9211" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38433 } } peers { permanent_uuid: "25a778364d7d42639e4b4224d5aafed8" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 41463 } } peers { permanent_uuid: "0ca8e269e476459ebdf315facaaa0ab5" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 35635 } }
I20260806 21:12:33.640760 844 heartbeater.cc:503] Master 127.16.178.62:43275 was elected leader, sending a full tablet report...
I20260806 21:12:33.641158 1308 ts_tablet_manager.cc:1428] T e474ca118b1941e69ad9fb4964dcf8c5 P 25a778364d7d42639e4b4224d5aafed8: Time spent starting tablet: real 0.006s user 0.002s sys 0.006s
W20260806 21:12:33.815102 1282 tablet.cc:2369] T e474ca118b1941e69ad9fb4964dcf8c5 P 0ca8e269e476459ebdf315facaaa0ab5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 21:12:33.842118 1059 tablet.cc:2369] T e474ca118b1941e69ad9fb4964dcf8c5 P 39d51b25a89d4580b56046bd767e9211: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 21:12:33.844921 1317 raft_consensus.cc:482] T e474ca118b1941e69ad9fb4964dcf8c5 P 25a778364d7d42639e4b4224d5aafed8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:12:33.845129 1317 raft_consensus.cc:504] T e474ca118b1941e69ad9fb4964dcf8c5 P 25a778364d7d42639e4b4224d5aafed8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d51b25a89d4580b56046bd767e9211" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38433 } } peers { permanent_uuid: "25a778364d7d42639e4b4224d5aafed8" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 41463 } } peers { permanent_uuid: "0ca8e269e476459ebdf315facaaa0ab5" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 35635 } }
I20260806 21:12:33.846258 1317 leader_election.cc:288] T e474ca118b1941e69ad9fb4964dcf8c5 P 25a778364d7d42639e4b4224d5aafed8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39d51b25a89d4580b56046bd767e9211 (127.16.178.2:38433), 0ca8e269e476459ebdf315facaaa0ab5 (127.16.178.3:35635)
I20260806 21:12:33.850381 1300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:33.846671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:47407 (local address 127.16.178.2:38433)
0806 21:12:33.846880 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:33.846884 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:33.847215 (+ 331us) server_negotiation.cc:407] Connection header received
0806 21:12:33.847343 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:33.847346 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:33.847398 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:33.847467 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:12:33.847706 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:33.848270 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:33.849472 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:33.849849 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:33.849955 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:33.850012 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:33.850185 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 21:12:33.850253 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260806 21:12:33.850410 1304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:33.846695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:35191 (local address 127.16.178.3:35635)
0806 21:12:33.846859 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:33.846863 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:33.847120 (+ 257us) server_negotiation.cc:407] Connection header received
0806 21:12:33.847255 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:33.847257 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:33.847301 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:33.847373 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:12:33.847652 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:33.848290 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:33.849451 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:33.849848 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:33.849955 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:33.850023 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:33.850214 (+ 191us) server_negotiation.cc:299] Negotiation successful
0806 21:12:33.850283 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 21:12:33.850420 1318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:33.846567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.2:38433 (local address 127.16.178.1:47407)
0806 21:12:33.847130 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:33.847141 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:33.847241 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:33.847486 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:33.847489 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:33.847504 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:12:33.847610 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:33.847615 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:33.848375 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:33.848379 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:33.849353 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:33.849363 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:33.849957 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:33.849960 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:33.850072 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:33.850093 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 21:12:33.850159 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 21:12:33.850241 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"mutex_wait_us":114,"thread_start_us":155,"threads_started":1}
I20260806 21:12:33.850426 1319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:33.846565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.3:35635 (local address 127.16.178.1:35191)
0806 21:12:33.847024 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:33.847036 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 21:12:33.847153 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:12:33.847394 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:12:33.847397 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:12:33.847413 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:12:33.847552 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:33.847558 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:33.848395 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:33.848399 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:33.849328 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:12:33.849338 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:12:33.849959 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:12:33.849963 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:12:33.850100 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:33.850120 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 21:12:33.850185 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 21:12:33.850260 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":398,"mutex_wait_us":56,"thread_start_us":166,"threads_started":1}
I20260806 21:12:33.851055 1202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e474ca118b1941e69ad9fb4964dcf8c5" candidate_uuid: "25a778364d7d42639e4b4224d5aafed8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ca8e269e476459ebdf315facaaa0ab5" is_pre_election: true
I20260806 21:12:33.851346 974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e474ca118b1941e69ad9fb4964dcf8c5" candidate_uuid: "25a778364d7d42639e4b4224d5aafed8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39d51b25a89d4580b56046bd767e9211" is_pre_election: true
I20260806 21:12:33.851387 1202 raft_consensus.cc:2454] T e474ca118b1941e69ad9fb4964dcf8c5 P 0ca8e269e476459ebdf315facaaa0ab5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25a778364d7d42639e4b4224d5aafed8 in term 0.
I20260806 21:12:33.851611 974 raft_consensus.cc:2454] T e474ca118b1941e69ad9fb4964dcf8c5 P 39d51b25a89d4580b56046bd767e9211 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25a778364d7d42639e4b4224d5aafed8 in term 0.
I20260806 21:12:33.851879 672 leader_election.cc:302] T e474ca118b1941e69ad9fb4964dcf8c5 P 25a778364d7d42639e4b4224d5aafed8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ca8e269e476459ebdf315facaaa0ab5, 25a778364d7d42639e4b4224d5aafed8; no voters:
I20260806 21:12:33.852178 1317 raft_consensus.cc:2781] T e474ca118b1941e69ad9fb4964dcf8c5 P 25a778364d7d42639e4b4224d5aafed8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:12:33.852275 1317 raft_consensus.cc:482] T e474ca118b1941e69ad9fb4964dcf8c5 P 25a778364d7d42639e4b4224d5aafed8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:12:33.852325 1317 raft_consensus.cc:3037] T e474ca118b1941e69ad9fb4964dcf8c5 P 25a778364d7d42639e4b4224d5aafed8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:12:33.853415 1317 raft_consensus.cc:504] T e474ca118b1941e69ad9fb4964dcf8c5 P 25a778364d7d42639e4b4224d5aafed8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d51b25a89d4580b56046bd767e9211" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38433 } } peers { permanent_uuid: "25a778364d7d42639e4b4224d5aafed8" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 41463 } } peers { permanent_uuid: "0ca8e269e476459ebdf315facaaa0ab5" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 35635 } }
W20260806 21:12:33.854085 847 tablet.cc:2369] T e474ca118b1941e69ad9fb4964dcf8c5 P 25a778364d7d42639e4b4224d5aafed8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 21:12:33.854200 974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e474ca118b1941e69ad9fb4964dcf8c5" candidate_uuid: "25a778364d7d42639e4b4224d5aafed8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39d51b25a89d4580b56046bd767e9211"
I20260806 21:12:33.854338 1202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e474ca118b1941e69ad9fb4964dcf8c5" candidate_uuid: "25a778364d7d42639e4b4224d5aafed8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ca8e269e476459ebdf315facaaa0ab5"
I20260806 21:12:33.854341 974 raft_consensus.cc:3037] T e474ca118b1941e69ad9fb4964dcf8c5 P 39d51b25a89d4580b56046bd767e9211 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:12:33.854456 1202 raft_consensus.cc:3037] T e474ca118b1941e69ad9fb4964dcf8c5 P 0ca8e269e476459ebdf315facaaa0ab5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:12:33.853857 1317 leader_election.cc:288] T e474ca118b1941e69ad9fb4964dcf8c5 P 25a778364d7d42639e4b4224d5aafed8 [CANDIDATE]: Term 1 election: Requested vote from peers 39d51b25a89d4580b56046bd767e9211 (127.16.178.2:38433), 0ca8e269e476459ebdf315facaaa0ab5 (127.16.178.3:35635)
I20260806 21:12:33.855733 1202 raft_consensus.cc:2454] T e474ca118b1941e69ad9fb4964dcf8c5 P 0ca8e269e476459ebdf315facaaa0ab5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25a778364d7d42639e4b4224d5aafed8 in term 1.
I20260806 21:12:33.855733 974 raft_consensus.cc:2454] T e474ca118b1941e69ad9fb4964dcf8c5 P 39d51b25a89d4580b56046bd767e9211 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25a778364d7d42639e4b4224d5aafed8 in term 1.
I20260806 21:12:33.856127 672 leader_election.cc:302] T e474ca118b1941e69ad9fb4964dcf8c5 P 25a778364d7d42639e4b4224d5aafed8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ca8e269e476459ebdf315facaaa0ab5, 25a778364d7d42639e4b4224d5aafed8; no voters:
I20260806 21:12:33.856312 1317 raft_consensus.cc:2781] T e474ca118b1941e69ad9fb4964dcf8c5 P 25a778364d7d42639e4b4224d5aafed8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:12:33.856585 1317 raft_consensus.cc:686] T e474ca118b1941e69ad9fb4964dcf8c5 P 25a778364d7d42639e4b4224d5aafed8 [term 1 LEADER]: Becoming Leader. State: Replica: 25a778364d7d42639e4b4224d5aafed8, State: Running, Role: LEADER
I20260806 21:12:33.856916 1317 consensus_queue.cc:237] T e474ca118b1941e69ad9fb4964dcf8c5 P 25a778364d7d42639e4b4224d5aafed8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d51b25a89d4580b56046bd767e9211" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38433 } } peers { permanent_uuid: "25a778364d7d42639e4b4224d5aafed8" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 41463 } } peers { permanent_uuid: "0ca8e269e476459ebdf315facaaa0ab5" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 35635 } }
I20260806 21:12:33.860169 562 catalog_manager.cc:5495] T e474ca118b1941e69ad9fb4964dcf8c5 P 25a778364d7d42639e4b4224d5aafed8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 25a778364d7d42639e4b4224d5aafed8 (127.16.178.1). New cstate: current_term: 1 leader_uuid: "25a778364d7d42639e4b4224d5aafed8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39d51b25a89d4580b56046bd767e9211" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25a778364d7d42639e4b4224d5aafed8" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ca8e269e476459ebdf315facaaa0ab5" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 35635 } health_report { overall_health: UNKNOWN } } }
I20260806 21:12:33.906461 1301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:33.902908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50016 (local address 127.16.178.1:41463)
0806 21:12:33.903065 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 21:12:33.903070 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:12:33.903084 (+ 14us) server_negotiation.cc:407] Connection header received
0806 21:12:33.903170 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:12:33.903173 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:12:33.903230 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:12:33.903319 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:12:33.903590 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:33.904113 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:33.904979 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:12:33.905200 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:12:33.905291 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:12:33.905608 (+ 317us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:12:33.906067 (+ 459us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:12:33.906179 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 21:12:33.906255 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 21:12:33.906319 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 21:12:33.912551 1202 raft_consensus.cc:1261] T e474ca118b1941e69ad9fb4964dcf8c5 P 0ca8e269e476459ebdf315facaaa0ab5 [term 1 FOLLOWER]: Refusing update from remote peer 25a778364d7d42639e4b4224d5aafed8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:12:33.912557 974 raft_consensus.cc:1261] T e474ca118b1941e69ad9fb4964dcf8c5 P 39d51b25a89d4580b56046bd767e9211 [term 1 FOLLOWER]: Refusing update from remote peer 25a778364d7d42639e4b4224d5aafed8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:12:33.913319 1320 consensus_queue.cc:1035] T e474ca118b1941e69ad9fb4964dcf8c5 P 25a778364d7d42639e4b4224d5aafed8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39d51b25a89d4580b56046bd767e9211" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:12:33.913506 1317 consensus_queue.cc:1035] T e474ca118b1941e69ad9fb4964dcf8c5 P 25a778364d7d42639e4b4224d5aafed8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ca8e269e476459ebdf315facaaa0ab5" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 35635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:12:33.921243 1325 mvcc.cc:205] Tried to move back new op lower bound from 7315663888018165760 to 7315663887800614912. Current Snapshot: MvccSnapshot[applied={T|T < 7315663888018165760}]
I20260806 21:12:33.925330 1330 mvcc.cc:205] Tried to move back new op lower bound from 7315663888018165760 to 7315663887800614912. Current Snapshot: MvccSnapshot[applied={T|T < 7315663888018165760}]
I20260806 21:12:33.931452 563 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50872:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:12:33.931625 563 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50872:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:12:33.934139 563 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 319f57158e304af9af4a3b3143797615: Sending DeleteTablet for 3 replicas of tablet e474ca118b1941e69ad9fb4964dcf8c5
I20260806 21:12:33.935093 947 tablet_service.cc:1514] Processing DeleteTablet for tablet e474ca118b1941e69ad9fb4964dcf8c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:12:33 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:50326
I20260806 21:12:33.935139 746 tablet_service.cc:1514] Processing DeleteTablet for tablet e474ca118b1941e69ad9fb4964dcf8c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:12:33 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:50002
I20260806 21:12:33.935214 1174 tablet_service.cc:1514] Processing DeleteTablet for tablet e474ca118b1941e69ad9fb4964dcf8c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:12:33 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:45944
I20260806 21:12:33.935915 1345 tablet_replica.cc:330] T e474ca118b1941e69ad9fb4964dcf8c5 P 25a778364d7d42639e4b4224d5aafed8: stopping tablet replica
I20260806 21:12:33.936271 1341 tablet_replica.cc:330] T e474ca118b1941e69ad9fb4964dcf8c5 P 39d51b25a89d4580b56046bd767e9211: stopping tablet replica
I20260806 21:12:33.936409 1345 raft_consensus.cc:2229] T e474ca118b1941e69ad9fb4964dcf8c5 P 25a778364d7d42639e4b4224d5aafed8 [term 1 LEADER]: Raft consensus shutting down.
I20260806 21:12:33.936434 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 652
I20260806 21:12:33.936578 1341 raft_consensus.cc:2229] T e474ca118b1941e69ad9fb4964dcf8c5 P 39d51b25a89d4580b56046bd767e9211 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:12:33.939050 1341 raft_consensus.cc:2258] T e474ca118b1941e69ad9fb4964dcf8c5 P 39d51b25a89d4580b56046bd767e9211 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:12:33.939143 1342 tablet_replica.cc:330] T e474ca118b1941e69ad9fb4964dcf8c5 P 0ca8e269e476459ebdf315facaaa0ab5: stopping tablet replica
I20260806 21:12:33.939461 1342 raft_consensus.cc:2229] T e474ca118b1941e69ad9fb4964dcf8c5 P 0ca8e269e476459ebdf315facaaa0ab5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:12:33.939841 1342 raft_consensus.cc:2258] T e474ca118b1941e69ad9fb4964dcf8c5 P 0ca8e269e476459ebdf315facaaa0ab5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:12:33.943380 1341 ts_tablet_manager.cc:1905] T e474ca118b1941e69ad9fb4964dcf8c5 P 39d51b25a89d4580b56046bd767e9211: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:12:33.944691 1342 ts_tablet_manager.cc:1905] T e474ca118b1941e69ad9fb4964dcf8c5 P 0ca8e269e476459ebdf315facaaa0ab5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:12:33.946105 1341 ts_tablet_manager.cc:1918] T e474ca118b1941e69ad9fb4964dcf8c5 P 39d51b25a89d4580b56046bd767e9211: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:12:33.946206 1341 log.cc:1196] T e474ca118b1941e69ad9fb4964dcf8c5 P 39d51b25a89d4580b56046bd767e9211: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-1/wal/wals/e474ca118b1941e69ad9fb4964dcf8c5
I20260806 21:12:33.946478 1341 ts_tablet_manager.cc:1939] T e474ca118b1941e69ad9fb4964dcf8c5 P 39d51b25a89d4580b56046bd767e9211: Deleting consensus metadata
I20260806 21:12:33.947427 1342 ts_tablet_manager.cc:1918] T e474ca118b1941e69ad9fb4964dcf8c5 P 0ca8e269e476459ebdf315facaaa0ab5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:12:33.947533 1342 log.cc:1196] T e474ca118b1941e69ad9fb4964dcf8c5 P 0ca8e269e476459ebdf315facaaa0ab5: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786050686797304-17096-0/minicluster-data/ts-2/wal/wals/e474ca118b1941e69ad9fb4964dcf8c5
I20260806 21:12:33.947850 1342 ts_tablet_manager.cc:1939] T e474ca118b1941e69ad9fb4964dcf8c5 P 0ca8e269e476459ebdf315facaaa0ab5: Deleting consensus metadata
I20260806 21:12:33.947786 544 catalog_manager.cc:4841] TS 39d51b25a89d4580b56046bd767e9211 (127.16.178.2:38433): tablet e474ca118b1941e69ad9fb4964dcf8c5 (table test-table [id=1e2b5a69183f48fdb7e737ea0aef2449]) successfully deleted
I20260806 21:12:33.949306 541 catalog_manager.cc:4841] TS 0ca8e269e476459ebdf315facaaa0ab5 (127.16.178.3:35635): tablet e474ca118b1941e69ad9fb4964dcf8c5 (table test-table [id=1e2b5a69183f48fdb7e737ea0aef2449]) successfully deleted
W20260806 21:12:33.952533 541 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.178.1:41463 (error 108)
I20260806 21:12:33.952593 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 852
I20260806 21:12:33.956123 1298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:12:33.955769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:41463 (local address 127.0.0.1:50018)
0806 21:12:33.955911 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 21:12:33.955991 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.1:41463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 21:12:33.956311 541 catalog_manager.cc:4584] TS 25a778364d7d42639e4b4224d5aafed8 (127.16.178.1:41463): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e474ca118b1941e69ad9fb4964dcf8c5: Network error: Client connection negotiation failed: client connection to 127.16.178.1:41463: connect: Connection refused (error 111)
I20260806 21:12:33.963153 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 1063
I20260806 21:12:33.981117 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 516
2026-08-06T21:12:33Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2648 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:12:34 dist-test-slave-lgr5 krb5kdc[1352](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:12:34 dist-test-slave-lgr5 krb5kdc[1352](info): set up 2 sockets
Aug 06 21:12:34 dist-test-slave-lgr5 krb5kdc[1352](info): commencing operation
krb5kdc: starting...
W20260806 21:12:35.124742 17096 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Aug 06 21:12:35 dist-test-slave-lgr5 krb5kdc[1352](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050755, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:12:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:12:35Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.16.178.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.16.178.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.16.178.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/krb5kdc/rangeradmin_127.16.178.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.16.178.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/krb5kdc/rangeradmin_127.16.178.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.16.178.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.16.178.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.16.178.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/krb5kdc/rangerlookup_127.16.178.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.16.178.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/krb5kdc/rangerlookup_127.16.178.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.16.178.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.16.178.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.16.178.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/krb5kdc/HTTP_127.16.178.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.16.178.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/krb5kdc/HTTP_127.16.178.20@KRBTEST.COM.keytab.
I20260806 21:12:35.307600 17096 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskJ1T3_n/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/postgres -l logfile start
2026-08-06 21:12:37.825 UTC [1391] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 21:12:37.825 UTC [1391] LOG: listening on IPv4 address "127.16.178.20", port 46033
2026-08-06 21:12:37.829 UTC [1391] LOG: listening on Unix socket "/tmp/.s.PGSQL.46033"
2026-08-06 21:12:37.851 UTC [1398] LOG: database system was shut down at 2026-08-06 21:12:36 UTC
2026-08-06 21:12:37.856 UTC [1391] LOG: database system is ready to accept connections
I20260806 21:12:38.909135 17096 mini_postgres.cc:96] Postgres bound to 46033
2026-08-06 21:12:38.918 UTC [1411] FATAL: database "slave" does not exist
127.16.178.20:46033 - accepting connections
I20260806 21:12:38.919036 17096 mini_ranger.cc:161] Starting Ranger...
I20260806 21:12:38.937069 17096 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 21:12:39.424784 17096 mini_ranger.cc:87] Created ranger Postgres database
I20260806 21:12:39.424880 17096 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ranger-admin
2026-08-06 21:12:39,734 [I] DB FLAVOR :POSTGRES
2026-08-06 21:12:39,734 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 21:12:39,734 [I] Checking connection..
2026-08-06 21:12:39,734 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.178.20:46033/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 21:12:40,011 [I] Checking connection passed.
2026-08-06 21:12:40,012 [I] --------- Verifying version history table ---------
2026-08-06 21:12:40,012 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.178.20:46033/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 21:12:40,264 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 21:12:40,264 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.178.20:46033/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 21:12:40,535 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 21:12:40,535 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 21:12:40,535 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.178.20:46033/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 21:12:40.782 UTC [1521] WARNING: there is no transaction in progress
2026-08-06 21:12:40,791 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 21:12:40,791 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.178.20:46033/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 21:12:41,057 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 21:12:41,057 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 21:12:41,057 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.178.20:46033/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 21:12:41,304 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.178.20:46033/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 21:12:41,571 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.178.20:46033/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-lgr5.c.gcp-upstream.internal','N') ;"
2026-08-06 21:12:41,831 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.178.20:46033/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 21:12:42,109 [I] Table x_portal_user does not exist in database ranger
2026-08-06 21:12:42,110 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.178.20:46033/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 21:12:42,370 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 21:12:42,371 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.178.20:46033/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 21:12:42,642 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.178.20:46033/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 21:12:42,922 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 21:12:42,922 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.178.20:46033/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 21:12:43.964 UTC [1692] WARNING: there is no transaction in progress
2026-08-06 21:12:43.980 UTC [1692] WARNING: there is no transaction in progress
2026-08-06 21:12:43.997 UTC [1692] WARNING: there is no transaction in progress
2026-08-06 21:12:44.015 UTC [1692] WARNING: there is no transaction in progress
2026-08-06 21:12:44.031 UTC [1692] WARNING: there is no transaction in progress
2026-08-06 21:12:44.050 UTC [1692] WARNING: there is no transaction in progress
2026-08-06 21:12:44.106 UTC [1692] WARNING: there is no transaction in progress
2026-08-06 21:12:44.114 UTC [1692] WARNING: there is no transaction in progress
2026-08-06 21:12:44.129 UTC [1692] WARNING: there is no transaction in progress
2026-08-06 21:12:44.140 UTC [1692] WARNING: there is no transaction in progress
2026-08-06 21:12:44.150 UTC [1692] WARNING: there is no transaction in progress
2026-08-06 21:12:44.164 UTC [1692] WARNING: there is no transaction in progress
2026-08-06 21:12:44.174 UTC [1692] WARNING: there is no transaction in progress
2026-08-06 21:12:44.181 UTC [1692] WARNING: there is no transaction in progress
2026-08-06 21:12:44.189 UTC [1692] WARNING: there is no transaction in progress
2026-08-06 21:12:44.197 UTC [1692] WARNING: there is no transaction in progress
2026-08-06 21:12:44,900 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 21:12:44,900 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.178.20:46033/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 21:12:45,149 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 21:12:45,150 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.178.20:46033/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 21:12:45,421 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 21:12:45,422 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.178.20:46033/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 21:12:45,698 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 21:12:45,699 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.178.20:46033/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 21:12:45,973 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.178.20:46033/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 21:12:46,223 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.178.20:46033/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-lgr5.c.gcp-upstream.internal';"
2026-08-06 21:12:46,500 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 21:12:46,500 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.178.20:46033/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 21:12:46,815 [I] DB_PATCHES have already been applied
I20260806 21:12:46.825084 17096 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ranger-admin:/tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskJ1T3_n/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 21:12:46.825212 17096 mini_ranger.cc:193] Using host: 127.16.178.20
I20260806 21:12:46.832656 17096 mini_ranger.cc:227] Ranger admin URL: http://127.16.178.20:52827
Aug 06, 2026 9:12:47 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 9:12:47 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 9:12:47 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 9:12:47 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 9:12:47 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 9:12:48 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 9:12:48 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 21:12:49.750733 17096 mini_ranger.cc:160] Time spent starting Ranger: real 10.832s user 0.008s sys 0.012s
Aug 06, 2026 9:13:09 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 9:13:09 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 06, 2026 9:13:09 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 9:13:09 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 9:13:09 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 9:13:09 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 9:13:09 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 9:13:09 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 9:13:09 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 9:13:09 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 9:13:09 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 9:13:09 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 9:13:09 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 9:13:09 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 9:13:09 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 9:13:09 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 9:13:09 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 9:13:10 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 21:13:11.146812 17096 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.16.178.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.16.178.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.16.178.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/krb5kdc/rangerkms_127.16.178.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.16.178.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/krb5kdc/rangerkms_127.16.178.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.16.178.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.16.178.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.16.178.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/krb5kdc/HTTP_127.16.178.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.16.178.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/krb5kdc/HTTP_127.16.178.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 21:13:11 dist-test-slave-lgr5 krb5kdc[1352](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050791, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 21:13:11.241374 17096 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 21:13:11.252576 17096 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 21:13:12.111653 17096 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 21:13:12.111738 17096 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ranger-kms
I20260806 21:13:12.111805 17096 mini_ranger_kms.cc:227] Using postgres at 127.16.178.20:46033
2026-08-06 21:13:12,424 [I] DB FLAVOR :POSTGRES
2026-08-06 21:13:12,425 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 21:13:12,425 [I] Checking connection
2026-08-06 21:13:12,425 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.178.20:46033/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 21:13:12,684 [I] connection success
2026-08-06 21:13:12,684 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 21:13:12,684 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 21:13:12,684 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.178.20:46033/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 21:13:12,943 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 21:13:12,943 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 21:13:12,944 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 21:13:12,944 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.178.20:46033/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 21:13:13,241 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 21:13:13.808928 17096 mini_ranger_kms.cc:310] Created kms service
I20260806 21:13:14.005438 17096 mini_ranger_kms.cc:326] Created kudu user
I20260806 21:13:14.055404 17096 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 21:13:14 dist-test-slave-lgr5 krb5kdc[1352](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050791, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.16.178.20@KRBTEST.COM
I20260806 21:13:14.501425 17096 mini_ranger_kms.cc:384] Added ranger policy
I20260806 21:13:14.501767 17096 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ranger-kms:/tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 21:13:14.501814 17096 mini_ranger_kms.cc:240] Using host: 127.16.178.20
I20260806 21:13:14.506858 17096 mini_ranger_kms.cc:276] Ranger KMS PID: 1994
I20260806 21:13:14.507037 17096 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.16.178.20:40345
Aug 06, 2026 9:13:15 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 9:13:15 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 9:13:15 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 9:13:15 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 9:13:16 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 9:13:16 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 21:13:17 dist-test-slave-lgr5 krb5kdc[1352](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786050797, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.16.178.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:13:17.106063 17096 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.865s user 0.020s sys 0.271s
I20260806 21:13:17.106218 17096 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 21:13:17.106335 17096 mini_ranger_kms.cc:401] 127.16.178.20:40345/kms/v1/keys
Aug 06 21:13:17 dist-test-slave-lgr5 krb5kdc[1352](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050797, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.16.178.20@KRBTEST.COM for HTTP/127.16.178.20@KRBTEST.COM
Aug 06 21:13:17 dist-test-slave-lgr5 krb5kdc[1352](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050797, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.16.178.20@KRBTEST.COM for HTTP/127.16.178.20@KRBTEST.COM
Aug 06 21:13:20 dist-test-slave-lgr5 krb5kdc[1352](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050800, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.16.178.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
I20260806 21:13:20.295540 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:43735
--webserver_interface=127.16.178.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:44113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:43735
--ranger_config_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.178.20:40345/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:13:20.416903 2068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:13:20.417219 2068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:13:20.417260 2068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:13:20.421511 2068 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:13:20.421595 2068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:13:20.421614 2068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:13:20.421629 2068 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:13:20.421643 2068 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:13:20.427901 2068 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:44113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/master-0/wal
--ranger_kms_url=127.16.178.20:40345/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:43735
--ranger_config_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:43735
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.178.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:13:20.429221 2068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:13:20.430418 2068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 21:13:20.440917 2068 server_base.cc:1034] running on GCE node
W20260806 21:13:20.440959 2075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:13:20.440968 2080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:13:20.441215 2077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:13:20.441845 2068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:13:20.443169 2068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:13:20.444368 2068 hybrid_clock.cc:648] HybridClock initialized: now 1786050800444348 us; error 35 us; skew 500 ppm
Aug 06 21:13:20 dist-test-slave-lgr5 krb5kdc[1352](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050800, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:13:20.447230 2068 init.cc:377] Logged in from keytab as kudu/127.16.178.62@KRBTEST.COM (short username kudu)
I20260806 21:13:20.448252 2068 webserver.cc:460] Webserver started at http://127.16.178.62:35953/ using document root <none> and password file <none>
I20260806 21:13:20.448815 2068 fs_manager.cc:359] Metadata directory not provided
I20260806 21:13:20.448876 2068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:13:20.449088 2068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 21:13:20 dist-test-slave-lgr5 krb5kdc[1352](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050800, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for HTTP/127.16.178.20@KRBTEST.COM
I20260806 21:13:20.511180 2068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "a4e64b899e2c4e48b4c83156628a22fa"
format_stamp: "Formatted at 2026-08-06 21:13:20 on dist-test-slave-lgr5"
server_key: "a2c9b3e77c514fd4830953e6a899f3c3"
server_key_iv: "617ecaeeea8ba1fb8b907a121b73208b"
server_key_version: "kuduclusterkey@0"
I20260806 21:13:20.512190 2068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "a4e64b899e2c4e48b4c83156628a22fa"
format_stamp: "Formatted at 2026-08-06 21:13:20 on dist-test-slave-lgr5"
server_key: "a2c9b3e77c514fd4830953e6a899f3c3"
server_key_iv: "617ecaeeea8ba1fb8b907a121b73208b"
server_key_version: "kuduclusterkey@0"
I20260806 21:13:20.516959 2068 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 21:13:20.537398 2092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:13:20.538828 2068 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 21:13:20.538971 2068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "a4e64b899e2c4e48b4c83156628a22fa"
format_stamp: "Formatted at 2026-08-06 21:13:20 on dist-test-slave-lgr5"
server_key: "a2c9b3e77c514fd4830953e6a899f3c3"
server_key_iv: "617ecaeeea8ba1fb8b907a121b73208b"
server_key_version: "kuduclusterkey@0"
I20260806 21:13:20.539112 2068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:13:20.556181 2068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:13:20.559342 2068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:13:20.559542 2068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:13:20.567978 2068 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.62:43735
I20260806 21:13:20.568001 2185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.62:43735 every 8 connection(s)
I20260806 21:13:20.569142 2068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:13:20.572197 2186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:13:20.575382 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 2068
I20260806 21:13:20.575534 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:13:20.575860 17096 external_mini_cluster.cc:1428] Setting key 88e399cd567b65fea92379cc82b3d9e9
I20260806 21:13:20.579941 2186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4e64b899e2c4e48b4c83156628a22fa: Bootstrap starting.
I20260806 21:13:20.582867 2186 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a4e64b899e2c4e48b4c83156628a22fa: Neither blocks nor log segments found. Creating new log.
I20260806 21:13:20.583753 2186 log.cc:824] T 00000000000000000000000000000000 P a4e64b899e2c4e48b4c83156628a22fa: Log is configured to *not* fsync() on all Append() calls
Aug 06 21:13:20 dist-test-slave-lgr5 krb5kdc[1352](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050800, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:13:20.586140 2186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4e64b899e2c4e48b4c83156628a22fa: No bootstrap required, opened a new log
I20260806 21:13:20.591141 2186 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4e64b899e2c4e48b4c83156628a22fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4e64b899e2c4e48b4c83156628a22fa" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 43735 } }
I20260806 21:13:20.591481 2186 raft_consensus.cc:374] T 00000000000000000000000000000000 P a4e64b899e2c4e48b4c83156628a22fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:13:20.591552 2186 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4e64b899e2c4e48b4c83156628a22fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4e64b899e2c4e48b4c83156628a22fa, State: Initialized, Role: FOLLOWER
I20260806 21:13:20.592147 2186 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4e64b899e2c4e48b4c83156628a22fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4e64b899e2c4e48b4c83156628a22fa" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 43735 } }
I20260806 21:13:20.592330 2186 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4e64b899e2c4e48b4c83156628a22fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:13:20.592334 2190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:13:20.577648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43346 (local address 127.16.178.62:43735)
0806 21:13:20.578273 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0806 21:13:20.578291 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 21:13:20.578330 (+ 39us) server_negotiation.cc:407] Connection header received
0806 21:13:20.579046 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:13:20.579065 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:13:20.579445 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:13:20.579755 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:13:20.580801 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:13:20.581829 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:13:20.583046 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:13:20.583610 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:13:20.583738 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:13:20.586396 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:13:20.586404 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:13:20.586425 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:13:20.586459 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:13:20.588584 (+ 2125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:13:20.589140 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:13:20.589147 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:13:20.589155 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:13:20.589323 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:13:20.589627 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:13:20.589631 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:13:20.589633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:13:20.590306 (+ 673us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:13:20.590560 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0806 21:13:20.590846 (+ 286us) server_negotiation.cc:299] Negotiation successful
0806 21:13:20.591195 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":169,"threads_started":1}
I20260806 21:13:20.592420 2186 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4e64b899e2c4e48b4c83156628a22fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:13:20.592523 2186 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4e64b899e2c4e48b4c83156628a22fa [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:13:20.593844 2186 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4e64b899e2c4e48b4c83156628a22fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4e64b899e2c4e48b4c83156628a22fa" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 43735 } }
I20260806 21:13:20.594211 2186 leader_election.cc:302] T 00000000000000000000000000000000 P a4e64b899e2c4e48b4c83156628a22fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4e64b899e2c4e48b4c83156628a22fa; no voters:
I20260806 21:13:20.594543 2186 leader_election.cc:288] T 00000000000000000000000000000000 P a4e64b899e2c4e48b4c83156628a22fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:13:20.594648 2192 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4e64b899e2c4e48b4c83156628a22fa [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:13:20.594899 2192 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4e64b899e2c4e48b4c83156628a22fa [term 1 LEADER]: Becoming Leader. State: Replica: a4e64b899e2c4e48b4c83156628a22fa, State: Running, Role: LEADER
I20260806 21:13:20.595317 2192 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4e64b899e2c4e48b4c83156628a22fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4e64b899e2c4e48b4c83156628a22fa" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 43735 } }
I20260806 21:13:20.595928 2186 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4e64b899e2c4e48b4c83156628a22fa [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:13:20.597100 2194 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4e64b899e2c4e48b4c83156628a22fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4e64b899e2c4e48b4c83156628a22fa. Latest consensus state: current_term: 1 leader_uuid: "a4e64b899e2c4e48b4c83156628a22fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4e64b899e2c4e48b4c83156628a22fa" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 43735 } } }
I20260806 21:13:20.597242 2194 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4e64b899e2c4e48b4c83156628a22fa [sys.catalog]: This master's current role is: LEADER
I20260806 21:13:20.597553 2193 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4e64b899e2c4e48b4c83156628a22fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a4e64b899e2c4e48b4c83156628a22fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4e64b899e2c4e48b4c83156628a22fa" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 43735 } } }
I20260806 21:13:20.597645 2193 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4e64b899e2c4e48b4c83156628a22fa [sys.catalog]: This master's current role is: LEADER
I20260806 21:13:20.597654 2208 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:13:20.601061 2208 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:13:20.602070 2186 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 21:13:20.610944 2208 catalog_manager.cc:1347] Generated new cluster ID: 15ac899574dc4e8db2f1f6439e1df424
I20260806 21:13:20.611038 2208 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:13:20.624425 2208 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:13:20.625895 2208 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:13:20.638978 2208 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a4e64b899e2c4e48b4c83156628a22fa: Generated new TSK 0
I20260806 21:13:20.640301 2208 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.178.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab.
Entry for principal kudu/127.16.178.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab.
I20260806 21:13:21.379701 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.178.20:40345/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.1:0
--local_ip_for_outbound_sockets=127.16.178.1
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:43735
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab
--principal=kudu/127.16.178.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:44113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:13:21.510102 2254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:13:21.510419 2254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:13:21.510483 2254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:13:21.514892 2254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:13:21.514993 2254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:13:21.515097 2254 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.1
I20260806 21:13:21.521001 2254 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:44113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.16.178.20:40345/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab
--principal=kudu/127.16.178.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:43735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2254
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.1
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:13:21.522256 2254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:13:21.523378 2254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:13:21.534732 2264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:13:21.534688 2267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:13:21.535130 2262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:13:21.536880 2254 server_base.cc:1034] running on GCE node
I20260806 21:13:21.537510 2254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:13:21.538509 2254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:13:21.541838 2254 hybrid_clock.cc:648] HybridClock initialized: now 1786050801541786 us; error 49 us; skew 500 ppm
Aug 06 21:13:21 dist-test-slave-lgr5 krb5kdc[1352](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050801, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:13:21.545385 2254 init.cc:377] Logged in from keytab as kudu/127.16.178.1@KRBTEST.COM (short username kudu)
I20260806 21:13:21.546516 2254 webserver.cc:460] Webserver started at http://127.16.178.1:43359/ using document root <none> and password file <none>
I20260806 21:13:21.547149 2254 fs_manager.cc:359] Metadata directory not provided
I20260806 21:13:21.547219 2254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:13:21.547442 2254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 21:13:21 dist-test-slave-lgr5 krb5kdc[1352](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050801, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for HTTP/127.16.178.20@KRBTEST.COM
I20260806 21:13:21.566988 2254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-0/data/instance:
uuid: "a642019bca814b1791cd6ad76a05b2a0"
format_stamp: "Formatted at 2026-08-06 21:13:21 on dist-test-slave-lgr5"
server_key: "354cdc79703f544c980065e0727c8732"
server_key_iv: "6895701032cf3d3dcd4284678f01bd98"
server_key_version: "kuduclusterkey@0"
I20260806 21:13:21.567987 2254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance:
uuid: "a642019bca814b1791cd6ad76a05b2a0"
format_stamp: "Formatted at 2026-08-06 21:13:21 on dist-test-slave-lgr5"
server_key: "354cdc79703f544c980065e0727c8732"
server_key_iv: "6895701032cf3d3dcd4284678f01bd98"
server_key_version: "kuduclusterkey@0"
I20260806 21:13:21.578001 2254 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.005s sys 0.000s
I20260806 21:13:21.598538 2278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:13:21.599957 2254 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:13:21.600124 2254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-0/wal
uuid: "a642019bca814b1791cd6ad76a05b2a0"
format_stamp: "Formatted at 2026-08-06 21:13:21 on dist-test-slave-lgr5"
server_key: "354cdc79703f544c980065e0727c8732"
server_key_iv: "6895701032cf3d3dcd4284678f01bd98"
server_key_version: "kuduclusterkey@0"
I20260806 21:13:21.600270 2254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:13:21.615649 2254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:13:21.619725 2254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:13:21.620011 2254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:13:21.620872 2254 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:13:21.622042 2254 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:13:21.622159 2254 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:13:21.622288 2254 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:13:21.622380 2254 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:13:21.637136 2254 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.1:44429
I20260806 21:13:21.638489 2254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
I20260806 21:13:21.641163 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 2254
I20260806 21:13:21.641289 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance
I20260806 21:13:21.641587 17096 external_mini_cluster.cc:1428] Setting key 1f66f6535a157e66b22a4fca5856ad18
I20260806 21:13:21.640344 2465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.1:44429 every 8 connection(s)
Aug 06 21:13:21 dist-test-slave-lgr5 krb5kdc[1352](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050801, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.178.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.2@KRBTEST.COM" created.
I20260806 21:13:21.666148 2472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:13:21.641064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:39899 (local address 127.16.178.62:43735)
0806 21:13:21.641324 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0806 21:13:21.641329 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:13:21.642252 (+ 923us) server_negotiation.cc:407] Connection header received
0806 21:13:21.643354 (+ 1102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:13:21.643358 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:13:21.643422 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:13:21.643517 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:13:21.644895 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:13:21.645760 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:13:21.647385 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:13:21.647740 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:13:21.647859 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:13:21.661193 (+ 13334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:13:21.661197 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:13:21.661200 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:13:21.661239 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:13:21.663220 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:13:21.664013 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:13:21.664017 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:13:21.664020 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:13:21.664143 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:13:21.664640 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:13:21.664644 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:13:21.664645 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:13:21.664897 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:13:21.664971 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 21:13:21.665809 (+ 838us) server_negotiation.cc:299] Negotiation successful
0806 21:13:21.665956 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":67,"threads_started":1}
I20260806 21:13:21.667918 2477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:13:21.641513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:43735 (local address 127.16.178.1:39899)
0806 21:13:21.642092 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0806 21:13:21.642141 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 21:13:21.643112 (+ 971us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:13:21.643716 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:13:21.643725 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:13:21.644056 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:13:21.644728 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:13:21.644746 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:13:21.645909 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:13:21.645912 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:13:21.647252 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:13:21.647264 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:13:21.647919 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:13:21.647924 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:13:21.648108 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:13:21.648751 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:13:21.648771 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:13:21.660871 (+ 12100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:13:21.663435 (+ 2564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:13:21.663443 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:13:21.663467 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:13:21.663884 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:13:21.664269 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:13:21.664272 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:13:21.664274 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:13:21.664518 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:13:21.665062 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:13:21.665069 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:13:21.665489 (+ 420us) client_negotiation.cc:769] Sending connection context
0806 21:13:21.666383 (+ 894us) client_negotiation.cc:240] Negotiation successful
0806 21:13:21.666788 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":152,"threads_started":1}
I20260806 21:13:21.669641 2466 heartbeater.cc:347] Connected to a master server at 127.16.178.62:43735
I20260806 21:13:21.669924 2466 heartbeater.cc:464] Registering TS with master...
I20260806 21:13:21.670545 2466 heartbeater.cc:511] Master 127.16.178.62:43735 requested a full tablet report, sending...
I20260806 21:13:21.672616 2114 ts_manager.cc:194] Registered new tserver with Master: a642019bca814b1791cd6ad76a05b2a0 (127.16.178.1:44429)
I20260806 21:13:21.674279 2114 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.1@KRBTEST.COM'} at 127.16.178.1:39899
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab.
Entry for principal kudu/127.16.178.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab.
I20260806 21:13:21.691737 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.178.20:40345/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.2:0
--local_ip_for_outbound_sockets=127.16.178.2
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:43735
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab
--principal=kudu/127.16.178.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:44113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 21:13:21 dist-test-slave-lgr5 krb5kdc[1352](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786050801, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 21:13:21.867514 2483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:13:21.868477 2483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:13:21.868628 2483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:13:21.875069 2483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:13:21.875313 2483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:13:21.875479 2483 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.2
I20260806 21:13:21.883050 2483 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:44113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.16.178.20:40345/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab
--principal=kudu/127.16.178.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:43735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2483
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.2
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:13:21.884514 2483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:13:21.885748 2483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:13:21.900048 2493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:13:21.900622 2495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:13:21.900882 2492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:13:21.902189 2483 server_base.cc:1034] running on GCE node
I20260806 21:13:21.902818 2483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:13:21.903929 2483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:13:21.907253 2483 hybrid_clock.cc:648] HybridClock initialized: now 1786050801906743 us; error 525 us; skew 500 ppm
Aug 06 21:13:21 dist-test-slave-lgr5 krb5kdc[1352](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050801, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:13:21.911595 2483 init.cc:377] Logged in from keytab as kudu/127.16.178.2@KRBTEST.COM (short username kudu)
I20260806 21:13:21.913169 2483 webserver.cc:460] Webserver started at http://127.16.178.2:43573/ using document root <none> and password file <none>
I20260806 21:13:21.914121 2483 fs_manager.cc:359] Metadata directory not provided
I20260806 21:13:21.914310 2483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:13:21.914654 2483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 21:13:21 dist-test-slave-lgr5 krb5kdc[1352](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050801, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.2@KRBTEST.COM for HTTP/127.16.178.20@KRBTEST.COM
I20260806 21:13:21.934455 2483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-1/data/instance:
uuid: "4b683a426bf14af8b37964f31e737aa6"
format_stamp: "Formatted at 2026-08-06 21:13:21 on dist-test-slave-lgr5"
server_key: "03fb6030ccadd012468d7b344516d0cd"
server_key_iv: "e0c5a1de0e8f03c89b40e45251420d4c"
server_key_version: "kuduclusterkey@0"
I20260806 21:13:21.935478 2483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance:
uuid: "4b683a426bf14af8b37964f31e737aa6"
format_stamp: "Formatted at 2026-08-06 21:13:21 on dist-test-slave-lgr5"
server_key: "03fb6030ccadd012468d7b344516d0cd"
server_key_iv: "e0c5a1de0e8f03c89b40e45251420d4c"
server_key_version: "kuduclusterkey@0"
I20260806 21:13:21.941491 2483 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 21:13:21.979454 2509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:13:21.989832 2483 fs_manager.cc:725] Time spent opening block manager: real 0.032s user 0.003s sys 0.001s
I20260806 21:13:21.990078 2483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-1/wal
uuid: "4b683a426bf14af8b37964f31e737aa6"
format_stamp: "Formatted at 2026-08-06 21:13:21 on dist-test-slave-lgr5"
server_key: "03fb6030ccadd012468d7b344516d0cd"
server_key_iv: "e0c5a1de0e8f03c89b40e45251420d4c"
server_key_version: "kuduclusterkey@0"
I20260806 21:13:21.990267 2483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:13:22.017210 2483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:13:22.021812 2483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:13:22.022157 2483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:13:22.023070 2483 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:13:22.024482 2483 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:13:22.024605 2483 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:13:22.024724 2483 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:13:22.024798 2483 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:13:22.038978 2483 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.2:39557
I20260806 21:13:22.040488 2483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
I20260806 21:13:22.042886 2690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.2:39557 every 8 connection(s)
I20260806 21:13:22.046765 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 2483
I20260806 21:13:22.046901 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance
I20260806 21:13:22.047233 17096 external_mini_cluster.cc:1428] Setting key 29d14a1ae687fa386ca7511e6f3cfae7
Aug 06 21:13:22 dist-test-slave-lgr5 krb5kdc[1352](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050801, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.2@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:13:22.067440 2472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:13:22.045091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:32885 (local address 127.16.178.62:43735)
0806 21:13:22.045248 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 21:13:22.045253 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:13:22.051991 (+ 6738us) server_negotiation.cc:407] Connection header received
0806 21:13:22.053426 (+ 1435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:13:22.053431 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:13:22.053494 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:13:22.053548 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:13:22.055462 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:13:22.056386 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:13:22.058222 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:13:22.058591 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:13:22.058681 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:13:22.062269 (+ 3588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:13:22.062274 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:13:22.062277 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:13:22.062310 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:13:22.064270 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:13:22.065176 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:13:22.065181 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:13:22.065183 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:13:22.065290 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:13:22.065807 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:13:22.065810 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:13:22.065812 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:13:22.066089 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:13:22.066171 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 21:13:22.067055 (+ 884us) server_negotiation.cc:299] Negotiation successful
0806 21:13:22.067232 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 21:13:22.069684 2699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:13:22.045633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:43735 (local address 127.16.178.2:32885)
0806 21:13:22.051782 (+ 6149us) negotiation.cc:107] Waiting for socket to connect
0806 21:13:22.051867 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0806 21:13:22.053106 (+ 1239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:13:22.054041 (+ 935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:13:22.054053 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:13:22.054492 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:13:22.055251 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:13:22.055274 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:13:22.056573 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:13:22.056577 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:13:22.058064 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:13:22.058078 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:13:22.058935 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:13:22.058940 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:13:22.059162 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:13:22.059995 (+ 833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:13:22.060019 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:13:22.062001 (+ 1982us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:13:22.064507 (+ 2506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:13:22.064516 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:13:22.064543 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:13:22.065023 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:13:22.065432 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:13:22.065435 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:13:22.065437 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:13:22.065681 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:13:22.066281 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:13:22.066290 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:13:22.066722 (+ 432us) client_negotiation.cc:769] Sending connection context
0806 21:13:22.067939 (+ 1217us) client_negotiation.cc:240] Negotiation successful
0806 21:13:22.068439 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5835,"thread_start_us":174,"threads_started":1}
I20260806 21:13:22.072721 2691 heartbeater.cc:347] Connected to a master server at 127.16.178.62:43735
I20260806 21:13:22.073103 2691 heartbeater.cc:464] Registering TS with master...
I20260806 21:13:22.074180 2691 heartbeater.cc:511] Master 127.16.178.62:43735 requested a full tablet report, sending...
I20260806 21:13:22.076004 2114 ts_manager.cc:194] Registered new tserver with Master: 4b683a426bf14af8b37964f31e737aa6 (127.16.178.2:39557)
I20260806 21:13:22.076784 2114 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.2@KRBTEST.COM'} at 127.16.178.2:32885
WARNING: no policy specified for kudu/127.16.178.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab.
Entry for principal kudu/127.16.178.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab.
I20260806 21:13:22.130544 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.178.20:40345/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.3:0
--local_ip_for_outbound_sockets=127.16.178.3
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:43735
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab
--principal=kudu/127.16.178.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:44113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 21:13:22 dist-test-slave-lgr5 krb5kdc[1352](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050801, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for HTTP/127.16.178.20@KRBTEST.COM
W20260806 21:13:22.335978 2704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:13:22.336364 2704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:13:22.336440 2704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:13:22.342501 2704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:13:22.342581 2704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:13:22.342665 2704 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.3
I20260806 21:13:22.351257 2704 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:44113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.16.178.20:40345/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab
--principal=kudu/127.16.178.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:43735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2704
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.3
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:13:22.353068 2704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:13:22.354396 2704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:13:22.369390 2719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:13:22.369437 2715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:13:22.369442 2716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:13:22.370087 2704 server_base.cc:1034] running on GCE node
I20260806 21:13:22.370676 2704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:13:22.371608 2704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:13:22.372915 2704 hybrid_clock.cc:648] HybridClock initialized: now 1786050802372826 us; error 110 us; skew 500 ppm
Aug 06 21:13:22 dist-test-slave-lgr5 krb5kdc[1352](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050802, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:13:22.377313 2704 init.cc:377] Logged in from keytab as kudu/127.16.178.3@KRBTEST.COM (short username kudu)
I20260806 21:13:22.378742 2704 webserver.cc:460] Webserver started at http://127.16.178.3:33165/ using document root <none> and password file <none>
I20260806 21:13:22.379614 2704 fs_manager.cc:359] Metadata directory not provided
I20260806 21:13:22.379809 2704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:13:22.380144 2704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 21:13:22 dist-test-slave-lgr5 krb5kdc[1352](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050802, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.3@KRBTEST.COM for HTTP/127.16.178.20@KRBTEST.COM
I20260806 21:13:22.407469 2704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-2/data/instance:
uuid: "42d64728154c4c50958460b296e3b1bf"
format_stamp: "Formatted at 2026-08-06 21:13:22 on dist-test-slave-lgr5"
server_key: "42c5b0c75c4511f0f197262def65ced1"
server_key_iv: "3a582232852c9e2d1ac9115c55033aea"
server_key_version: "kuduclusterkey@0"
I20260806 21:13:22.408388 2704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance:
uuid: "42d64728154c4c50958460b296e3b1bf"
format_stamp: "Formatted at 2026-08-06 21:13:22 on dist-test-slave-lgr5"
server_key: "42c5b0c75c4511f0f197262def65ced1"
server_key_iv: "3a582232852c9e2d1ac9115c55033aea"
server_key_version: "kuduclusterkey@0"
I20260806 21:13:22.428159 2704 fs_manager.cc:691] Time spent creating directory manager: real 0.015s user 0.005s sys 0.000s
I20260806 21:13:22.469579 2731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:13:22.471657 2704 fs_manager.cc:725] Time spent opening block manager: real 0.021s user 0.004s sys 0.000s
I20260806 21:13:22.471851 2704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-2/wal
uuid: "42d64728154c4c50958460b296e3b1bf"
format_stamp: "Formatted at 2026-08-06 21:13:22 on dist-test-slave-lgr5"
server_key: "42c5b0c75c4511f0f197262def65ced1"
server_key_iv: "3a582232852c9e2d1ac9115c55033aea"
server_key_version: "kuduclusterkey@0"
I20260806 21:13:22.472028 2704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:13:22.499183 2704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:13:22.503935 2704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:13:22.504295 2704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:13:22.505196 2704 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:13:22.506682 2704 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:13:22.506846 2704 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:13:22.506991 2704 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:13:22.507117 2704 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:13:22.523368 2704 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.3:42243
I20260806 21:13:22.525629 2704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
I20260806 21:13:22.527990 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 2704
I20260806 21:13:22.528167 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance
I20260806 21:13:22.528548 17096 external_mini_cluster.cc:1428] Setting key 68ef9aed766f3bdadbbd0c07c54fe4fb
I20260806 21:13:22.527187 2907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.3:42243 every 8 connection(s)
Aug 06 21:13:22 dist-test-slave-lgr5 krb5kdc[1352](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050802, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.3@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:13:22.552649 2472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:13:22.534756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:34933 (local address 127.16.178.62:43735)
0806 21:13:22.534914 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 21:13:22.534920 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:13:22.535656 (+ 736us) server_negotiation.cc:407] Connection header received
0806 21:13:22.537103 (+ 1447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:13:22.537107 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:13:22.537168 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:13:22.537221 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:13:22.539022 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:13:22.539892 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:13:22.543735 (+ 3843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:13:22.544162 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:13:22.544244 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:13:22.547661 (+ 3417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:13:22.547666 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:13:22.547669 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:13:22.547703 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:13:22.549529 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:13:22.550443 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:13:22.550452 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:13:22.550455 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:13:22.550579 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:13:22.551086 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:13:22.551090 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:13:22.551092 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:13:22.551338 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:13:22.551413 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 21:13:22.552311 (+ 898us) server_negotiation.cc:299] Negotiation successful
0806 21:13:22.552457 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 21:13:22.557126 2911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:13:22.533457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:43735 (local address 127.16.178.3:34933)
0806 21:13:22.535370 (+ 1913us) negotiation.cc:107] Waiting for socket to connect
0806 21:13:22.535525 (+ 155us) client_negotiation.cc:174] Beginning negotiation
0806 21:13:22.536802 (+ 1277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:13:22.537687 (+ 885us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:13:22.537697 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:13:22.538103 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:13:22.538816 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:13:22.538838 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:13:22.540070 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:13:22.540074 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:13:22.543581 (+ 3507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:13:22.543599 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:13:22.544488 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:13:22.544492 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:13:22.544734 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:13:22.545536 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:13:22.545559 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:13:22.547428 (+ 1869us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:13:22.549693 (+ 2265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:13:22.549701 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:13:22.549721 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:13:22.550243 (+ 522us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:13:22.550735 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:13:22.550738 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:13:22.550740 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:13:22.550955 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:13:22.551522 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:13:22.551530 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:13:22.551922 (+ 392us) client_negotiation.cc:769] Sending connection context
0806 21:13:22.553193 (+ 1271us) client_negotiation.cc:240] Negotiation successful
0806 21:13:22.555823 (+ 2630us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1654,"thread_start_us":206,"threads_started":1}
I20260806 21:13:22.558804 2908 heartbeater.cc:347] Connected to a master server at 127.16.178.62:43735
I20260806 21:13:22.559115 2908 heartbeater.cc:464] Registering TS with master...
I20260806 21:13:22.559782 2908 heartbeater.cc:511] Master 127.16.178.62:43735 requested a full tablet report, sending...
I20260806 21:13:22.561228 2114 ts_manager.cc:194] Registered new tserver with Master: 42d64728154c4c50958460b296e3b1bf (127.16.178.3:42243)
I20260806 21:13:22.562278 2114 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.3@KRBTEST.COM'} at 127.16.178.3:34933
I20260806 21:13:22.567605 17096 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 21:13:22.587481 2472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:13:22.576668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43356 (local address 127.16.178.62:43735)
0806 21:13:22.576813 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 21:13:22.576829 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 21:13:22.577638 (+ 809us) server_negotiation.cc:407] Connection header received
0806 21:13:22.577850 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:13:22.577853 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:13:22.577906 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:13:22.577958 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:13:22.578985 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:13:22.579795 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:13:22.581062 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:13:22.581394 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:13:22.581478 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:13:22.583459 (+ 1981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:13:22.583464 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:13:22.583467 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:13:22.583500 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:13:22.585266 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:13:22.585826 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:13:22.585830 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:13:22.585832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:13:22.585930 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:13:22.586278 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:13:22.586282 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:13:22.586283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:13:22.586524 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:13:22.586630 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 21:13:22.587130 (+ 500us) server_negotiation.cc:299] Negotiation successful
0806 21:13:22.587298 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 21:13:22.592034 2114 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43356:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:13:22.597802 2114 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:13:22.636873 2939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:13:22.613638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.2:39557 (local address 127.0.0.1:55268)
0806 21:13:22.619761 (+ 6123us) negotiation.cc:107] Waiting for socket to connect
0806 21:13:22.619851 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0806 21:13:22.620007 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:13:22.624185 (+ 4178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:13:22.624188 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:13:22.624209 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:13:22.624312 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:13:22.624318 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:13:22.627175 (+ 2857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:13:22.627178 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:13:22.628313 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:13:22.628323 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:13:22.636367 (+ 8044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:13:22.636372 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:13:22.636488 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:13:22.636517 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 21:13:22.636586 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 21:13:22.636675 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5891,"thread_start_us":168,"threads_started":1}
I20260806 21:13:22.637638 2936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:13:22.617508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:44429 (local address 127.0.0.1:54040)
0806 21:13:22.622398 (+ 4890us) negotiation.cc:107] Waiting for socket to connect
0806 21:13:22.622414 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:13:22.622513 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:13:22.623363 (+ 850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:13:22.623370 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:13:22.623396 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:13:22.623554 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:13:22.623560 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:13:22.630673 (+ 7113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:13:22.630676 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:13:22.631820 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:13:22.631842 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:13:22.633297 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:13:22.633300 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:13:22.633441 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:13:22.633490 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 21:13:22.633620 (+ 130us) client_negotiation.cc:240] Negotiation successful
0806 21:13:22.633776 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4822,"thread_start_us":87,"threads_started":1}
I20260806 21:13:22.638309 2937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:13:22.616986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36134 (local address 127.16.178.3:42243)
0806 21:13:22.624358 (+ 7372us) server_negotiation.cc:206] Beginning negotiation
0806 21:13:22.624373 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 21:13:22.624400 (+ 27us) server_negotiation.cc:407] Connection header received
0806 21:13:22.624478 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:13:22.624483 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:13:22.624687 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:13:22.624850 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:13:22.625262 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:13:22.626408 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:13:22.636248 (+ 9840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:13:22.637139 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:13:22.637253 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:13:22.637433 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 21:13:22.638036 (+ 603us) server_negotiation.cc:299] Negotiation successful
0806 21:13:22.638163 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7230,"thread_start_us":207,"threads_started":1}
I20260806 21:13:22.637919 2933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:13:22.616737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.3:42243 (local address 127.0.0.1:36134)
0806 21:13:22.620532 (+ 3795us) negotiation.cc:107] Waiting for socket to connect
0806 21:13:22.620547 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:13:22.620636 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:13:22.625009 (+ 4373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:13:22.625012 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:13:22.625031 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:13:22.625135 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:13:22.625142 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:13:22.626553 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:13:22.626557 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:13:22.628003 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:13:22.628016 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:13:22.637505 (+ 9489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:13:22.637509 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:13:22.637615 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:13:22.637640 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 21:13:22.637717 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 21:13:22.637805 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3724,"thread_start_us":130,"threads_started":1}
I20260806 21:13:22.638769 2940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:13:22.617971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54040 (local address 127.16.178.1:44429)
0806 21:13:22.620367 (+ 2396us) server_negotiation.cc:206] Beginning negotiation
0806 21:13:22.620376 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 21:13:22.622621 (+ 2245us) server_negotiation.cc:407] Connection header received
0806 21:13:22.622721 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:13:22.622727 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:13:22.623000 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:13:22.623162 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:13:22.624462 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:13:22.630531 (+ 6069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:13:22.631996 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:13:22.632926 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:13:22.633038 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:13:22.633225 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0806 21:13:22.638234 (+ 5009us) server_negotiation.cc:299] Negotiation successful
0806 21:13:22.638632 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2256,"thread_start_us":85,"threads_started":1}
I20260806 21:13:22.639294 2934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:13:22.614804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55268 (local address 127.16.178.2:39557)
0806 21:13:22.620909 (+ 6105us) server_negotiation.cc:206] Beginning negotiation
0806 21:13:22.620916 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 21:13:22.620946 (+ 30us) server_negotiation.cc:407] Connection header received
0806 21:13:22.621039 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:13:22.621045 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:13:22.623812 (+ 2767us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:13:22.623971 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:13:22.625897 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:13:22.627029 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:13:22.634961 (+ 7932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:13:22.635953 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:13:22.636100 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:13:22.636297 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0806 21:13:22.639024 (+ 2727us) server_negotiation.cc:299] Negotiation successful
0806 21:13:22.639148 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5970,"thread_start_us":111,"threads_started":1}
I20260806 21:13:22.642099 2803 tablet_service.cc:1467] Processing CreateTablet for tablet fb96080192694d70b305fbb4be9c1cca (DEFAULT_TABLE table=test-table [id=b5e6deea974d47959b3b61a17a240708]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:13:22.643471 2803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb96080192694d70b305fbb4be9c1cca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:13:22.646490 2358 tablet_service.cc:1467] Processing CreateTablet for tablet fb96080192694d70b305fbb4be9c1cca (DEFAULT_TABLE table=test-table [id=b5e6deea974d47959b3b61a17a240708]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:13:22.647733 2358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb96080192694d70b305fbb4be9c1cca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:13:22.653693 2941 tablet_bootstrap.cc:492] T fb96080192694d70b305fbb4be9c1cca P 42d64728154c4c50958460b296e3b1bf: Bootstrap starting.
I20260806 21:13:22.667390 2943 tablet_bootstrap.cc:492] T fb96080192694d70b305fbb4be9c1cca P a642019bca814b1791cd6ad76a05b2a0: Bootstrap starting.
I20260806 21:13:22.669009 2941 tablet_bootstrap.cc:654] T fb96080192694d70b305fbb4be9c1cca P 42d64728154c4c50958460b296e3b1bf: Neither blocks nor log segments found. Creating new log.
I20260806 21:13:22.670043 2941 log.cc:824] T fb96080192694d70b305fbb4be9c1cca P 42d64728154c4c50958460b296e3b1bf: Log is configured to *not* fsync() on all Append() calls
I20260806 21:13:22.671136 2943 tablet_bootstrap.cc:654] T fb96080192694d70b305fbb4be9c1cca P a642019bca814b1791cd6ad76a05b2a0: Neither blocks nor log segments found. Creating new log.
I20260806 21:13:22.672314 2943 log.cc:824] T fb96080192694d70b305fbb4be9c1cca P a642019bca814b1791cd6ad76a05b2a0: Log is configured to *not* fsync() on all Append() calls
I20260806 21:13:22.672919 2941 tablet_bootstrap.cc:492] T fb96080192694d70b305fbb4be9c1cca P 42d64728154c4c50958460b296e3b1bf: No bootstrap required, opened a new log
I20260806 21:13:22.673190 2941 ts_tablet_manager.cc:1397] T fb96080192694d70b305fbb4be9c1cca P 42d64728154c4c50958460b296e3b1bf: Time spent bootstrapping tablet: real 0.020s user 0.007s sys 0.000s
I20260806 21:13:22.675225 2943 tablet_bootstrap.cc:492] T fb96080192694d70b305fbb4be9c1cca P a642019bca814b1791cd6ad76a05b2a0: No bootstrap required, opened a new log
I20260806 21:13:22.674860 2576 tablet_service.cc:1467] Processing CreateTablet for tablet fb96080192694d70b305fbb4be9c1cca (DEFAULT_TABLE table=test-table [id=b5e6deea974d47959b3b61a17a240708]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:13:22.675496 2943 ts_tablet_manager.cc:1397] T fb96080192694d70b305fbb4be9c1cca P a642019bca814b1791cd6ad76a05b2a0: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260806 21:13:22.677507 2466 heartbeater.cc:503] Master 127.16.178.62:43735 was elected leader, sending a full tablet report...
I20260806 21:13:22.677626 2941 raft_consensus.cc:348] T fb96080192694d70b305fbb4be9c1cca P 42d64728154c4c50958460b296e3b1bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b683a426bf14af8b37964f31e737aa6" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39557 } } peers { permanent_uuid: "42d64728154c4c50958460b296e3b1bf" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42243 } } peers { permanent_uuid: "a642019bca814b1791cd6ad76a05b2a0" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 44429 } }
I20260806 21:13:22.677950 2941 raft_consensus.cc:374] T fb96080192694d70b305fbb4be9c1cca P 42d64728154c4c50958460b296e3b1bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:13:22.678006 2941 raft_consensus.cc:729] T fb96080192694d70b305fbb4be9c1cca P 42d64728154c4c50958460b296e3b1bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42d64728154c4c50958460b296e3b1bf, State: Initialized, Role: FOLLOWER
I20260806 21:13:22.678624 2941 consensus_queue.cc:260] T fb96080192694d70b305fbb4be9c1cca P 42d64728154c4c50958460b296e3b1bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b683a426bf14af8b37964f31e737aa6" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39557 } } peers { permanent_uuid: "42d64728154c4c50958460b296e3b1bf" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42243 } } peers { permanent_uuid: "a642019bca814b1791cd6ad76a05b2a0" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 44429 } }
I20260806 21:13:22.679857 2576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb96080192694d70b305fbb4be9c1cca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:13:22.685653 2943 raft_consensus.cc:348] T fb96080192694d70b305fbb4be9c1cca P a642019bca814b1791cd6ad76a05b2a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b683a426bf14af8b37964f31e737aa6" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39557 } } peers { permanent_uuid: "42d64728154c4c50958460b296e3b1bf" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42243 } } peers { permanent_uuid: "a642019bca814b1791cd6ad76a05b2a0" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 44429 } }
I20260806 21:13:22.686146 2943 raft_consensus.cc:374] T fb96080192694d70b305fbb4be9c1cca P a642019bca814b1791cd6ad76a05b2a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:13:22.686254 2943 raft_consensus.cc:729] T fb96080192694d70b305fbb4be9c1cca P a642019bca814b1791cd6ad76a05b2a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a642019bca814b1791cd6ad76a05b2a0, State: Initialized, Role: FOLLOWER
I20260806 21:13:22.686915 2943 consensus_queue.cc:260] T fb96080192694d70b305fbb4be9c1cca P a642019bca814b1791cd6ad76a05b2a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b683a426bf14af8b37964f31e737aa6" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39557 } } peers { permanent_uuid: "42d64728154c4c50958460b296e3b1bf" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42243 } } peers { permanent_uuid: "a642019bca814b1791cd6ad76a05b2a0" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 44429 } }
I20260806 21:13:22.688417 2943 ts_tablet_manager.cc:1428] T fb96080192694d70b305fbb4be9c1cca P a642019bca814b1791cd6ad76a05b2a0: Time spent starting tablet: real 0.013s user 0.007s sys 0.000s
I20260806 21:13:22.688640 2941 ts_tablet_manager.cc:1428] T fb96080192694d70b305fbb4be9c1cca P 42d64728154c4c50958460b296e3b1bf: Time spent starting tablet: real 0.015s user 0.007s sys 0.000s
I20260806 21:13:22.688985 2908 heartbeater.cc:503] Master 127.16.178.62:43735 was elected leader, sending a full tablet report...
I20260806 21:13:22.707995 2951 tablet_bootstrap.cc:492] T fb96080192694d70b305fbb4be9c1cca P 4b683a426bf14af8b37964f31e737aa6: Bootstrap starting.
I20260806 21:13:22.712111 2951 tablet_bootstrap.cc:654] T fb96080192694d70b305fbb4be9c1cca P 4b683a426bf14af8b37964f31e737aa6: Neither blocks nor log segments found. Creating new log.
I20260806 21:13:22.714154 2951 log.cc:824] T fb96080192694d70b305fbb4be9c1cca P 4b683a426bf14af8b37964f31e737aa6: Log is configured to *not* fsync() on all Append() calls
Aug 06 21:13:22 dist-test-slave-lgr5 krb5kdc[1352](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050801, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for HTTP/127.16.178.20@KRBTEST.COM
I20260806 21:13:22.723488 2951 tablet_bootstrap.cc:492] T fb96080192694d70b305fbb4be9c1cca P 4b683a426bf14af8b37964f31e737aa6: No bootstrap required, opened a new log
I20260806 21:13:22.723800 2951 ts_tablet_manager.cc:1397] T fb96080192694d70b305fbb4be9c1cca P 4b683a426bf14af8b37964f31e737aa6: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.000s
I20260806 21:13:22.729925 2951 raft_consensus.cc:348] T fb96080192694d70b305fbb4be9c1cca P 4b683a426bf14af8b37964f31e737aa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b683a426bf14af8b37964f31e737aa6" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39557 } } peers { permanent_uuid: "42d64728154c4c50958460b296e3b1bf" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42243 } } peers { permanent_uuid: "a642019bca814b1791cd6ad76a05b2a0" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 44429 } }
I20260806 21:13:22.732060 2951 raft_consensus.cc:374] T fb96080192694d70b305fbb4be9c1cca P 4b683a426bf14af8b37964f31e737aa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:13:22.732252 2951 raft_consensus.cc:729] T fb96080192694d70b305fbb4be9c1cca P 4b683a426bf14af8b37964f31e737aa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b683a426bf14af8b37964f31e737aa6, State: Initialized, Role: FOLLOWER
I20260806 21:13:22.733047 2951 consensus_queue.cc:260] T fb96080192694d70b305fbb4be9c1cca P 4b683a426bf14af8b37964f31e737aa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b683a426bf14af8b37964f31e737aa6" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39557 } } peers { permanent_uuid: "42d64728154c4c50958460b296e3b1bf" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42243 } } peers { permanent_uuid: "a642019bca814b1791cd6ad76a05b2a0" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 44429 } }
I20260806 21:13:22.734565 2951 ts_tablet_manager.cc:1428] T fb96080192694d70b305fbb4be9c1cca P 4b683a426bf14af8b37964f31e737aa6: Time spent starting tablet: real 0.011s user 0.005s sys 0.003s
I20260806 21:13:22.735080 2691 heartbeater.cc:503] Master 127.16.178.62:43735 was elected leader, sending a full tablet report...
W20260806 21:13:22.778116 2910 tablet.cc:2369] T fb96080192694d70b305fbb4be9c1cca P 42d64728154c4c50958460b296e3b1bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 21:13:22.788244 2953 raft_consensus.cc:482] T fb96080192694d70b305fbb4be9c1cca P 4b683a426bf14af8b37964f31e737aa6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:13:22.788554 2953 raft_consensus.cc:504] T fb96080192694d70b305fbb4be9c1cca P 4b683a426bf14af8b37964f31e737aa6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b683a426bf14af8b37964f31e737aa6" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39557 } } peers { permanent_uuid: "42d64728154c4c50958460b296e3b1bf" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42243 } } peers { permanent_uuid: "a642019bca814b1791cd6ad76a05b2a0" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 44429 } }
I20260806 21:13:22.790171 2953 leader_election.cc:288] T fb96080192694d70b305fbb4be9c1cca P 4b683a426bf14af8b37964f31e737aa6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 42d64728154c4c50958460b296e3b1bf (127.16.178.3:42243), a642019bca814b1791cd6ad76a05b2a0 (127.16.178.1:44429)
W20260806 21:13:22.795521 2694 tablet.cc:2369] T fb96080192694d70b305fbb4be9c1cca P 4b683a426bf14af8b37964f31e737aa6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 21:13:22.797130 2937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:13:22.791630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:45611 (local address 127.16.178.3:42243)
0806 21:13:22.791766 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 21:13:22.791771 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:13:22.792023 (+ 252us) server_negotiation.cc:407] Connection header received
0806 21:13:22.792275 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:13:22.792279 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:13:22.792330 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:13:22.792376 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:13:22.792886 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:13:22.793707 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:13:22.795544 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:13:22.796120 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:13:22.796226 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:13:22.796285 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 21:13:22.796892 (+ 607us) server_negotiation.cc:299] Negotiation successful
0806 21:13:22.796984 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 21:13:22.797897 2955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:13:22.790655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.3:42243 (local address 127.16.178.2:45611)
0806 21:13:22.791935 (+ 1280us) negotiation.cc:107] Waiting for socket to connect
0806 21:13:22.791950 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:13:22.792138 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:13:22.792556 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:13:22.792559 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:13:22.792596 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:13:22.792740 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:13:22.792747 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:13:22.793895 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:13:22.793898 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:13:22.795376 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:13:22.795393 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:13:22.796593 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:13:22.796597 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:13:22.796706 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:13:22.796732 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 21:13:22.797630 (+ 898us) client_negotiation.cc:240] Negotiation successful
0806 21:13:22.797726 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1209,"mutex_wait_us":38,"thread_start_us":117,"threads_started":1}
I20260806 21:13:22.798781 2811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb96080192694d70b305fbb4be9c1cca" candidate_uuid: "4b683a426bf14af8b37964f31e737aa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42d64728154c4c50958460b296e3b1bf" is_pre_election: true
I20260806 21:13:22.799165 2811 raft_consensus.cc:2454] T fb96080192694d70b305fbb4be9c1cca P 42d64728154c4c50958460b296e3b1bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b683a426bf14af8b37964f31e737aa6 in term 0.
I20260806 21:13:22.801070 2511 leader_election.cc:302] T fb96080192694d70b305fbb4be9c1cca P 4b683a426bf14af8b37964f31e737aa6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42d64728154c4c50958460b296e3b1bf, 4b683a426bf14af8b37964f31e737aa6; no voters:
I20260806 21:13:22.801487 2953 raft_consensus.cc:2781] T fb96080192694d70b305fbb4be9c1cca P 4b683a426bf14af8b37964f31e737aa6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:13:22.801594 2953 raft_consensus.cc:482] T fb96080192694d70b305fbb4be9c1cca P 4b683a426bf14af8b37964f31e737aa6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:13:22.801636 2953 raft_consensus.cc:3037] T fb96080192694d70b305fbb4be9c1cca P 4b683a426bf14af8b37964f31e737aa6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:13:22.803181 2953 raft_consensus.cc:504] T fb96080192694d70b305fbb4be9c1cca P 4b683a426bf14af8b37964f31e737aa6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b683a426bf14af8b37964f31e737aa6" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39557 } } peers { permanent_uuid: "42d64728154c4c50958460b296e3b1bf" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42243 } } peers { permanent_uuid: "a642019bca814b1791cd6ad76a05b2a0" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 44429 } }
I20260806 21:13:22.803653 2953 leader_election.cc:288] T fb96080192694d70b305fbb4be9c1cca P 4b683a426bf14af8b37964f31e737aa6 [CANDIDATE]: Term 1 election: Requested vote from peers 42d64728154c4c50958460b296e3b1bf (127.16.178.3:42243), a642019bca814b1791cd6ad76a05b2a0 (127.16.178.1:44429)
I20260806 21:13:22.804955 2811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb96080192694d70b305fbb4be9c1cca" candidate_uuid: "4b683a426bf14af8b37964f31e737aa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42d64728154c4c50958460b296e3b1bf"
I20260806 21:13:22.805131 2811 raft_consensus.cc:3037] T fb96080192694d70b305fbb4be9c1cca P 42d64728154c4c50958460b296e3b1bf [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:13:22.806665 2811 raft_consensus.cc:2454] T fb96080192694d70b305fbb4be9c1cca P 42d64728154c4c50958460b296e3b1bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b683a426bf14af8b37964f31e737aa6 in term 1.
I20260806 21:13:22.807292 2511 leader_election.cc:302] T fb96080192694d70b305fbb4be9c1cca P 4b683a426bf14af8b37964f31e737aa6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42d64728154c4c50958460b296e3b1bf, 4b683a426bf14af8b37964f31e737aa6; no voters:
I20260806 21:13:22.810168 2953 raft_consensus.cc:2781] T fb96080192694d70b305fbb4be9c1cca P 4b683a426bf14af8b37964f31e737aa6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:13:22.810570 2953 raft_consensus.cc:686] T fb96080192694d70b305fbb4be9c1cca P 4b683a426bf14af8b37964f31e737aa6 [term 1 LEADER]: Becoming Leader. State: Replica: 4b683a426bf14af8b37964f31e737aa6, State: Running, Role: LEADER
I20260806 21:13:22.811054 2953 consensus_queue.cc:237] T fb96080192694d70b305fbb4be9c1cca P 4b683a426bf14af8b37964f31e737aa6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b683a426bf14af8b37964f31e737aa6" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39557 } } peers { permanent_uuid: "42d64728154c4c50958460b296e3b1bf" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42243 } } peers { permanent_uuid: "a642019bca814b1791cd6ad76a05b2a0" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 44429 } }
I20260806 21:13:22.816501 2114 catalog_manager.cc:5495] T fb96080192694d70b305fbb4be9c1cca P 4b683a426bf14af8b37964f31e737aa6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b683a426bf14af8b37964f31e737aa6 (127.16.178.2). New cstate: current_term: 1 leader_uuid: "4b683a426bf14af8b37964f31e737aa6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b683a426bf14af8b37964f31e737aa6" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 39557 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42d64728154c4c50958460b296e3b1bf" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a642019bca814b1791cd6ad76a05b2a0" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 44429 } health_report { overall_health: UNKNOWN } } }
I20260806 21:13:22.816991 2940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:13:22.791717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:37615 (local address 127.16.178.1:44429)
0806 21:13:22.791882 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 21:13:22.791887 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:13:22.794762 (+ 2875us) server_negotiation.cc:407] Connection header received
0806 21:13:22.794976 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:13:22.794979 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:13:22.795062 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:13:22.795111 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:13:22.812471 (+ 17360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:13:22.813526 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:13:22.815343 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:13:22.816007 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:13:22.816196 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:13:22.816266 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 21:13:22.816715 (+ 449us) server_negotiation.cc:299] Negotiation successful
0806 21:13:22.816837 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 21:13:22.817552 2959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:13:22.791078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:44429 (local address 127.16.178.2:37615)
0806 21:13:22.794662 (+ 3584us) negotiation.cc:107] Waiting for socket to connect
0806 21:13:22.794678 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:13:22.794854 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:13:22.795248 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:13:22.795251 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:13:22.795269 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:13:22.812209 (+ 16940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:13:22.812233 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:13:22.813778 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:13:22.813781 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:13:22.815109 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:13:22.815127 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:13:22.816395 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:13:22.816399 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:13:22.816535 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:13:22.816569 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 21:13:22.817245 (+ 676us) client_negotiation.cc:240] Negotiation successful
0806 21:13:22.817357 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3500,"mutex_wait_us":3,"thread_start_us":115,"threads_started":1}
I20260806 21:13:22.818562 2380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb96080192694d70b305fbb4be9c1cca" candidate_uuid: "4b683a426bf14af8b37964f31e737aa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a642019bca814b1791cd6ad76a05b2a0" is_pre_election: true
I20260806 21:13:22.819041 2380 raft_consensus.cc:2454] T fb96080192694d70b305fbb4be9c1cca P a642019bca814b1791cd6ad76a05b2a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b683a426bf14af8b37964f31e737aa6 in term 0.
I20260806 21:13:22.819516 2381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb96080192694d70b305fbb4be9c1cca" candidate_uuid: "4b683a426bf14af8b37964f31e737aa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a642019bca814b1791cd6ad76a05b2a0"
I20260806 21:13:22.819675 2381 raft_consensus.cc:3037] T fb96080192694d70b305fbb4be9c1cca P a642019bca814b1791cd6ad76a05b2a0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:13:22.821619 2381 raft_consensus.cc:2454] T fb96080192694d70b305fbb4be9c1cca P a642019bca814b1791cd6ad76a05b2a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b683a426bf14af8b37964f31e737aa6 in term 1.
I20260806 21:13:22.852250 2934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:13:22.846246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55282 (local address 127.16.178.2:39557)
0806 21:13:22.846411 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 21:13:22.846415 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:13:22.846538 (+ 123us) server_negotiation.cc:407] Connection header received
0806 21:13:22.846729 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:13:22.846733 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:13:22.846822 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:13:22.846963 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:13:22.847409 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:13:22.848277 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:13:22.849647 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:13:22.850010 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:13:22.850112 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:13:22.850620 (+ 508us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:13:22.851254 (+ 634us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:13:22.851397 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 21:13:22.851967 (+ 570us) server_negotiation.cc:299] Negotiation successful
0806 21:13:22.852044 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 21:13:22.861267 2381 raft_consensus.cc:1261] T fb96080192694d70b305fbb4be9c1cca P a642019bca814b1791cd6ad76a05b2a0 [term 1 FOLLOWER]: Refusing update from remote peer 4b683a426bf14af8b37964f31e737aa6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:13:22.862743 2960 consensus_queue.cc:1035] T fb96080192694d70b305fbb4be9c1cca P 4b683a426bf14af8b37964f31e737aa6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a642019bca814b1791cd6ad76a05b2a0" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 44429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:13:22.872555 2811 raft_consensus.cc:1261] T fb96080192694d70b305fbb4be9c1cca P 42d64728154c4c50958460b296e3b1bf [term 1 FOLLOWER]: Refusing update from remote peer 4b683a426bf14af8b37964f31e737aa6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:13:22.873718 2960 consensus_queue.cc:1035] T fb96080192694d70b305fbb4be9c1cca P 4b683a426bf14af8b37964f31e737aa6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "42d64728154c4c50958460b296e3b1bf" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 42243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:13:22.899820 2964 mvcc.cc:205] Tried to move back new op lower bound from 7315664088506609664 to 7315664088321748992. Current Snapshot: MvccSnapshot[applied={T|T < 7315664088506609664}]
W20260806 21:13:22.899928 2467 tablet.cc:2369] T fb96080192694d70b305fbb4be9c1cca P a642019bca814b1791cd6ad76a05b2a0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 21:13:22.908782 2969 mvcc.cc:205] Tried to move back new op lower bound from 7315664088506609664 to 7315664088321748992. Current Snapshot: MvccSnapshot[applied={T|T < 7315664088506609664}]
I20260806 21:13:22.911128 2965 mvcc.cc:205] Tried to move back new op lower bound from 7315664088506609664 to 7315664088321748992. Current Snapshot: MvccSnapshot[applied={T|T < 7315664088506609664}]
I20260806 21:13:22.924958 2114 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:13:22.925176 2114 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:13:22.928828 2114 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a4e64b899e2c4e48b4c83156628a22fa: Sending DeleteTablet for 3 replicas of tablet fb96080192694d70b305fbb4be9c1cca
I20260806 21:13:22.930389 2803 tablet_service.cc:1514] Processing DeleteTablet for tablet fb96080192694d70b305fbb4be9c1cca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:13:22 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:36134
I20260806 21:13:22.930392 2358 tablet_service.cc:1514] Processing DeleteTablet for tablet fb96080192694d70b305fbb4be9c1cca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:13:22 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:54040
I20260806 21:13:22.931188 2979 tablet_replica.cc:330] T fb96080192694d70b305fbb4be9c1cca P 42d64728154c4c50958460b296e3b1bf: stopping tablet replica
I20260806 21:13:22.931226 2576 tablet_service.cc:1514] Processing DeleteTablet for tablet fb96080192694d70b305fbb4be9c1cca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:13:22 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:55268
I20260806 21:13:22.931730 2979 raft_consensus.cc:2229] T fb96080192694d70b305fbb4be9c1cca P 42d64728154c4c50958460b296e3b1bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:13:22.932838 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 2254
W20260806 21:13:22.950270 2093 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.178.1:44429 (error 108)
I20260806 21:13:22.951277 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 2483
I20260806 21:13:22.932179 2979 raft_consensus.cc:2258] T fb96080192694d70b305fbb4be9c1cca P 42d64728154c4c50958460b296e3b1bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:13:22.971838 2933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:13:22.971436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:44429 (local address 127.0.0.1:54052)
0806 21:13:22.971615 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 21:13:22.971723 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.1:44429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 21:13:22.971930 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 2704
I20260806 21:13:22.977963 2933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:13:22.977488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.2:39557 (local address 127.0.0.1:55290)
0806 21:13:22.977768 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0806 21:13:22.977846 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.2:39557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181}
W20260806 21:13:22.972046 2093 catalog_manager.cc:4584] TS a642019bca814b1791cd6ad76a05b2a0 (127.16.178.1:44429): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fb96080192694d70b305fbb4be9c1cca: Network error: Client connection negotiation failed: client connection to 127.16.178.1:44429: connect: Connection refused (error 111)
I20260806 21:13:23.015293 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 2068
I20260806 21:13:23.035288 17096 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 21:13:23.381422 17096 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 21:13:23.381567 17096 mini_ranger.cc:66] Stopping Ranger...
I20260806 21:13:23.729714 17096 mini_ranger.cc:68] Stopped Ranger
2026-08-06 21:13:23.729 UTC [1391] LOG: received smart shutdown request
2026-08-06 21:13:23.734 UTC [1391] LOG: background worker "logical replication launcher" (PID 1409) exited with exit code 1
2026-08-06 21:13:23.734 UTC [1399] LOG: shutting down
2026-08-06 21:13:23.925 UTC [1391] LOG: database system is shut down
2026-08-06T21:13:23Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50054 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:13:24 dist-test-slave-lgr5 krb5kdc[2994](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:13:24 dist-test-slave-lgr5 krb5kdc[2994](info): set up 2 sockets
Aug 06 21:13:24 dist-test-slave-lgr5 krb5kdc[2994](info): commencing operation
krb5kdc: starting...
W20260806 21:13:25.206086 17096 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Aug 06 21:13:25 dist-test-slave-lgr5 krb5kdc[2994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050805, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:13:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:13:25Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.16.178.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.16.178.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.16.178.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/rangeradmin_127.16.178.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.16.178.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/rangeradmin_127.16.178.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.16.178.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.16.178.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.16.178.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/rangerlookup_127.16.178.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.16.178.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/rangerlookup_127.16.178.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.16.178.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.16.178.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.16.178.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/HTTP_127.16.178.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.16.178.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/HTTP_127.16.178.20@KRBTEST.COM.keytab.
I20260806 21:13:25.396420 17096 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskJ1T3_n/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/postgres -l logfile start
2026-08-06 21:13:28.032 UTC [3042] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 21:13:28.032 UTC [3042] LOG: listening on IPv4 address "127.16.178.20", port 33847
2026-08-06 21:13:28.037 UTC [3042] LOG: listening on Unix socket "/tmp/.s.PGSQL.33847"
2026-08-06 21:13:28.066 UTC [3045] LOG: database system was shut down at 2026-08-06 21:13:26 UTC
2026-08-06 21:13:28.070 UTC [3042] LOG: database system is ready to accept connections
I20260806 21:13:29.127493 17096 mini_postgres.cc:96] Postgres bound to 33847
2026-08-06 21:13:29.134 UTC [3057] FATAL: database "slave" does not exist
127.16.178.20:33847 - accepting connections
I20260806 21:13:29.135564 17096 mini_ranger.cc:161] Starting Ranger...
I20260806 21:13:29.153061 17096 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 21:13:29.628684 17096 mini_ranger.cc:87] Created ranger Postgres database
I20260806 21:13:29.628762 17096 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ranger-admin
2026-08-06 21:13:29,910 [I] DB FLAVOR :POSTGRES
2026-08-06 21:13:29,911 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 21:13:29,911 [I] Checking connection..
2026-08-06 21:13:29,911 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.178.20:33847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 21:13:30,171 [I] Checking connection passed.
2026-08-06 21:13:30,171 [I] --------- Verifying version history table ---------
2026-08-06 21:13:30,171 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.178.20:33847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 21:13:30,435 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 21:13:30,436 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.178.20:33847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 21:13:30,697 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 21:13:30,697 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 21:13:30,697 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.178.20:33847/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 21:13:30.954 UTC [3170] WARNING: there is no transaction in progress
2026-08-06 21:13:30,962 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 21:13:30,963 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.178.20:33847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 21:13:31,246 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 21:13:31,246 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 21:13:31,246 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.178.20:33847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 21:13:31,506 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.178.20:33847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 21:13:31,774 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.178.20:33847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-lgr5.c.gcp-upstream.internal','N') ;"
2026-08-06 21:13:32,022 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.178.20:33847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 21:13:32,301 [I] Table x_portal_user does not exist in database ranger
2026-08-06 21:13:32,302 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.178.20:33847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 21:13:32,574 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 21:13:32,574 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.178.20:33847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 21:13:32,847 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.178.20:33847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 21:13:33,127 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 21:13:33,128 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.178.20:33847/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 21:13:34.185 UTC [3349] WARNING: there is no transaction in progress
2026-08-06 21:13:34.203 UTC [3349] WARNING: there is no transaction in progress
2026-08-06 21:13:34.220 UTC [3349] WARNING: there is no transaction in progress
2026-08-06 21:13:34.235 UTC [3349] WARNING: there is no transaction in progress
2026-08-06 21:13:34.251 UTC [3349] WARNING: there is no transaction in progress
2026-08-06 21:13:34.268 UTC [3349] WARNING: there is no transaction in progress
2026-08-06 21:13:34.332 UTC [3349] WARNING: there is no transaction in progress
2026-08-06 21:13:34.339 UTC [3349] WARNING: there is no transaction in progress
2026-08-06 21:13:34.354 UTC [3349] WARNING: there is no transaction in progress
2026-08-06 21:13:34.365 UTC [3349] WARNING: there is no transaction in progress
2026-08-06 21:13:34.375 UTC [3349] WARNING: there is no transaction in progress
2026-08-06 21:13:34.389 UTC [3349] WARNING: there is no transaction in progress
2026-08-06 21:13:34.399 UTC [3349] WARNING: there is no transaction in progress
2026-08-06 21:13:34.407 UTC [3349] WARNING: there is no transaction in progress
2026-08-06 21:13:34.414 UTC [3349] WARNING: there is no transaction in progress
2026-08-06 21:13:34.421 UTC [3349] WARNING: there is no transaction in progress
2026-08-06 21:13:35,091 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 21:13:35,092 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.178.20:33847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 21:13:35,343 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 21:13:35,343 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.178.20:33847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 21:13:35,613 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 21:13:35,613 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.178.20:33847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 21:13:35,877 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 21:13:35,878 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.178.20:33847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 21:13:36,161 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.178.20:33847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 21:13:36,423 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.178.20:33847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-lgr5.c.gcp-upstream.internal';"
2026-08-06 21:13:36,695 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 21:13:36,695 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.178.20:33847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 21:13:36,951 [I] DB_PATCHES have already been applied
I20260806 21:13:36.957909 17096 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ranger-admin:/tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskJ1T3_n/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 21:13:36.957993 17096 mini_ranger.cc:193] Using host: 127.16.178.20
I20260806 21:13:36.962808 17096 mini_ranger.cc:227] Ranger admin URL: http://127.16.178.20:39311
Aug 06, 2026 9:13:37 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 9:13:37 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 9:13:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 9:13:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 9:13:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 9:13:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 9:13:37 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 21:13:38.209007 17096 mini_ranger.cc:160] Time spent starting Ranger: real 9.073s user 0.000s sys 0.013s
Aug 06, 2026 9:13:59 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 9:13:59 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 9:13:59 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 9:13:59 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 9:13:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 9:13:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 9:13:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 9:13:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 9:13:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 9:13:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 9:13:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 9:13:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 9:13:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 9:13:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 9:13:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 9:13:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 9:13:59 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 9:14:00 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 21:14:00.803829 17096 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.16.178.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.16.178.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.16.178.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/rangerkms_127.16.178.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.16.178.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/rangerkms_127.16.178.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.16.178.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.16.178.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.16.178.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/HTTP_127.16.178.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.16.178.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/HTTP_127.16.178.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 21:14:00 dist-test-slave-lgr5 krb5kdc[2994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050840, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 21:14:00.896136 17096 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 21:14:00.907785 17096 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 21:14:01.815083 17096 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 21:14:01.815173 17096 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ranger-kms
I20260806 21:14:01.815189 17096 mini_ranger_kms.cc:227] Using postgres at 127.16.178.20:33847
2026-08-06 21:14:01,997 [I] DB FLAVOR :POSTGRES
2026-08-06 21:14:01,997 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 21:14:01,997 [I] Checking connection
2026-08-06 21:14:01,997 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.178.20:33847/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 21:14:02,246 [I] connection success
2026-08-06 21:14:02,246 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 21:14:02,246 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 21:14:02,246 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.178.20:33847/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 21:14:02,504 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 21:14:02,504 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 21:14:02,504 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 21:14:02,504 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.178.20:33847/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 21:14:02,775 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 21:14:03.271080 17096 mini_ranger_kms.cc:310] Created kms service
I20260806 21:14:03.461567 17096 mini_ranger_kms.cc:326] Created kudu user
I20260806 21:14:03.515370 17096 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 21:14:03 dist-test-slave-lgr5 krb5kdc[2994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050840, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.16.178.20@KRBTEST.COM
I20260806 21:14:04.034618 17096 mini_ranger_kms.cc:384] Added ranger policy
I20260806 21:14:04.034812 17096 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ranger-kms:/tmp/dist-test-taskJ1T3_n/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskJ1T3_n/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 21:14:04.034834 17096 mini_ranger_kms.cc:240] Using host: 127.16.178.20
I20260806 21:14:04.040249 17096 mini_ranger_kms.cc:276] Ranger KMS PID: 3647
I20260806 21:14:04.040344 17096 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.16.178.20:60901
Aug 06, 2026 9:14:05 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 9:14:05 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 9:14:05 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 9:14:05 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 9:14:05 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 9:14:05 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 21:14:06 dist-test-slave-lgr5 krb5kdc[2994](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786050846, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.16.178.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:14:06.577988 17096 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.682s user 0.018s sys 0.147s
I20260806 21:14:06.578128 17096 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 21:14:06.578197 17096 mini_ranger_kms.cc:401] 127.16.178.20:60901/kms/v1/keys
Aug 06 21:14:06 dist-test-slave-lgr5 krb5kdc[2994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050846, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.16.178.20@KRBTEST.COM for HTTP/127.16.178.20@KRBTEST.COM
Aug 06 21:14:07 dist-test-slave-lgr5 krb5kdc[2994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050846, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.16.178.20@KRBTEST.COM for HTTP/127.16.178.20@KRBTEST.COM
Aug 06 21:14:09 dist-test-slave-lgr5 krb5kdc[2994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050849, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.16.178.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
I20260806 21:14:09.784305 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:35317
--webserver_interface=127.16.178.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:40385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:35317
--ranger_config_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.178.20:60901/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:14:09.909441 3729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:14:09.909741 3729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:14:09.909780 3729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:14:09.914021 3729 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:14:09.914100 3729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:14:09.914119 3729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:14:09.914134 3729 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:14:09.914149 3729 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:14:09.919978 3729 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:40385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/master-0/wal
--ranger_kms_url=127.16.178.20:60901/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:35317
--ranger_config_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:35317
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.178.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:14:09.921236 3729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:14:09.922333 3729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 21:14:09.932020 3729 server_base.cc:1034] running on GCE node
W20260806 21:14:09.932236 3742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:14:09.932299 3737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:14:09.932312 3738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:14:09.933238 3729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:14:09.934645 3729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:14:09.935875 3729 hybrid_clock.cc:648] HybridClock initialized: now 1786050849935842 us; error 55 us; skew 500 ppm
Aug 06 21:14:09 dist-test-slave-lgr5 krb5kdc[2994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050849, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:14:09.942373 3729 init.cc:377] Logged in from keytab as kudu/127.16.178.62@KRBTEST.COM (short username kudu)
I20260806 21:14:09.943598 3729 webserver.cc:460] Webserver started at http://127.16.178.62:44213/ using document root <none> and password file <none>
I20260806 21:14:09.944285 3729 fs_manager.cc:359] Metadata directory not provided
I20260806 21:14:09.944352 3729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:14:09.944597 3729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 21:14:09 dist-test-slave-lgr5 krb5kdc[2994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050849, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for HTTP/127.16.178.20@KRBTEST.COM
I20260806 21:14:10.009060 3729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "1f1abc66f1744b849522d675d88595bd"
format_stamp: "Formatted at 2026-08-06 21:14:10 on dist-test-slave-lgr5"
server_key: "a1be7fdfccccfaacf5c07756601d7b2d"
server_key_iv: "0c475cf161ebb1ae0c0a5aff00fb3054"
server_key_version: "kuduclusterkey@0"
I20260806 21:14:10.009796 3729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "1f1abc66f1744b849522d675d88595bd"
format_stamp: "Formatted at 2026-08-06 21:14:10 on dist-test-slave-lgr5"
server_key: "a1be7fdfccccfaacf5c07756601d7b2d"
server_key_iv: "0c475cf161ebb1ae0c0a5aff00fb3054"
server_key_version: "kuduclusterkey@0"
I20260806 21:14:10.014055 3729 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:14:10.032924 3754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:10.034719 3729 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 21:14:10.034910 3729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "1f1abc66f1744b849522d675d88595bd"
format_stamp: "Formatted at 2026-08-06 21:14:10 on dist-test-slave-lgr5"
server_key: "a1be7fdfccccfaacf5c07756601d7b2d"
server_key_iv: "0c475cf161ebb1ae0c0a5aff00fb3054"
server_key_version: "kuduclusterkey@0"
I20260806 21:14:10.035094 3729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:14:10.097980 3729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:14:10.101028 3729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:14:10.101226 3729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:14:10.109100 3729 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.62:35317
I20260806 21:14:10.109145 3840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.62:35317 every 8 connection(s)
I20260806 21:14:10.110116 3729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:14:10.113767 3842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:14:10.114714 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 3729
I20260806 21:14:10.114827 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:14:10.115116 17096 external_mini_cluster.cc:1428] Setting key 8b9455f5e6e6d086dfea5d7c4a375107
I20260806 21:14:10.121194 3842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f1abc66f1744b849522d675d88595bd: Bootstrap starting.
Aug 06 21:14:10 dist-test-slave-lgr5 krb5kdc[2994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050849, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:14:10.124145 3842 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f1abc66f1744b849522d675d88595bd: Neither blocks nor log segments found. Creating new log.
I20260806 21:14:10.125064 3842 log.cc:824] T 00000000000000000000000000000000 P 1f1abc66f1744b849522d675d88595bd: Log is configured to *not* fsync() on all Append() calls
I20260806 21:14:10.127393 3842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f1abc66f1744b849522d675d88595bd: No bootstrap required, opened a new log
I20260806 21:14:10.129292 3851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:10.116594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58208 (local address 127.16.178.62:35317)
0806 21:14:10.117032 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:10.117049 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:10.117085 (+ 36us) server_negotiation.cc:407] Connection header received
0806 21:14:10.117766 (+ 681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:10.117791 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:10.118137 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:10.118442 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:14:10.119249 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:10.120290 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:10.121183 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:10.121690 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:10.121836 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:10.124214 (+ 2378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:14:10.124221 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:14:10.124238 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:14:10.124272 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:14:10.126411 (+ 2139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:10.126814 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:10.126820 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:10.126824 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:10.126928 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:10.127219 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:10.127223 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:10.127225 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:10.127711 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:14:10.127903 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:10.128202 (+ 299us) server_negotiation.cc:299] Negotiation successful
0806 21:14:10.128595 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":124,"threads_started":1}
I20260806 21:14:10.131496 3842 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f1abc66f1744b849522d675d88595bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f1abc66f1744b849522d675d88595bd" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 35317 } }
I20260806 21:14:10.131769 3842 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f1abc66f1744b849522d675d88595bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:14:10.131822 3842 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f1abc66f1744b849522d675d88595bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f1abc66f1744b849522d675d88595bd, State: Initialized, Role: FOLLOWER
I20260806 21:14:10.132309 3842 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f1abc66f1744b849522d675d88595bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f1abc66f1744b849522d675d88595bd" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 35317 } }
I20260806 21:14:10.132468 3842 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f1abc66f1744b849522d675d88595bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:14:10.132570 3842 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f1abc66f1744b849522d675d88595bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:14:10.132673 3842 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f1abc66f1744b849522d675d88595bd [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:14:10.133785 3842 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f1abc66f1744b849522d675d88595bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f1abc66f1744b849522d675d88595bd" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 35317 } }
I20260806 21:14:10.134157 3842 leader_election.cc:302] T 00000000000000000000000000000000 P 1f1abc66f1744b849522d675d88595bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f1abc66f1744b849522d675d88595bd; no voters:
I20260806 21:14:10.134523 3842 leader_election.cc:288] T 00000000000000000000000000000000 P 1f1abc66f1744b849522d675d88595bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:14:10.134613 3853 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f1abc66f1744b849522d675d88595bd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:14:10.134840 3853 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f1abc66f1744b849522d675d88595bd [term 1 LEADER]: Becoming Leader. State: Replica: 1f1abc66f1744b849522d675d88595bd, State: Running, Role: LEADER
I20260806 21:14:10.135210 3853 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f1abc66f1744b849522d675d88595bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f1abc66f1744b849522d675d88595bd" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 35317 } }
I20260806 21:14:10.135793 3842 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f1abc66f1744b849522d675d88595bd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:14:10.136803 3855 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f1abc66f1744b849522d675d88595bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f1abc66f1744b849522d675d88595bd. Latest consensus state: current_term: 1 leader_uuid: "1f1abc66f1744b849522d675d88595bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f1abc66f1744b849522d675d88595bd" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 35317 } } }
I20260806 21:14:10.136922 3855 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f1abc66f1744b849522d675d88595bd [sys.catalog]: This master's current role is: LEADER
I20260806 21:14:10.137252 3854 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f1abc66f1744b849522d675d88595bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f1abc66f1744b849522d675d88595bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f1abc66f1744b849522d675d88595bd" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 35317 } } }
I20260806 21:14:10.137327 3854 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f1abc66f1744b849522d675d88595bd [sys.catalog]: This master's current role is: LEADER
I20260806 21:14:10.137955 3842 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 21:14:10.138217 3869 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:14:10.144549 3869 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:14:10.158304 3869 catalog_manager.cc:1347] Generated new cluster ID: b1a3964411e244ecb39ed4706f74722b
I20260806 21:14:10.158407 3869 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:14:10.182617 3869 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:14:10.184274 3869 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:14:10.199151 3869 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f1abc66f1744b849522d675d88595bd: Generated new TSK 0
I20260806 21:14:10.200356 3869 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.178.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab.
Entry for principal kudu/127.16.178.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab.
I20260806 21:14:10.922767 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.178.20:60901/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.1:0
--local_ip_for_outbound_sockets=127.16.178.1
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:35317
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab
--principal=kudu/127.16.178.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:40385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:14:11.092979 3916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:14:11.093297 3916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:14:11.093361 3916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:14:11.097780 3916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:14:11.097875 3916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:14:11.097961 3916 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.1
I20260806 21:14:11.105618 3916 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:40385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.16.178.20:60901/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab
--principal=kudu/127.16.178.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:35317
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.1
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:14:11.107470 3916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:14:11.109082 3916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:14:11.126065 3932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:14:11.127012 3930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:14:11.127184 3935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:14:11.128221 3916 server_base.cc:1034] running on GCE node
I20260806 21:14:11.128996 3916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:14:11.130065 3916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:14:11.131338 3916 hybrid_clock.cc:648] HybridClock initialized: now 1786050851131280 us; error 68 us; skew 500 ppm
Aug 06 21:14:11 dist-test-slave-lgr5 krb5kdc[2994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050851, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:14:11.136281 3916 init.cc:377] Logged in from keytab as kudu/127.16.178.1@KRBTEST.COM (short username kudu)
I20260806 21:14:11.137979 3916 webserver.cc:460] Webserver started at http://127.16.178.1:45153/ using document root <none> and password file <none>
I20260806 21:14:11.138897 3916 fs_manager.cc:359] Metadata directory not provided
I20260806 21:14:11.139000 3916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:14:11.139313 3916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 21:14:11 dist-test-slave-lgr5 krb5kdc[2994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050851, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for HTTP/127.16.178.20@KRBTEST.COM
I20260806 21:14:11.163609 3916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-0/data/instance:
uuid: "5bbd400049864275a32fb933c1b99eb3"
format_stamp: "Formatted at 2026-08-06 21:14:11 on dist-test-slave-lgr5"
server_key: "2d5361f11b627db39c77afa43b80d51d"
server_key_iv: "7e1b526b03707bbaee3142c692aa0562"
server_key_version: "kuduclusterkey@0"
I20260806 21:14:11.164633 3916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance:
uuid: "5bbd400049864275a32fb933c1b99eb3"
format_stamp: "Formatted at 2026-08-06 21:14:11 on dist-test-slave-lgr5"
server_key: "2d5361f11b627db39c77afa43b80d51d"
server_key_iv: "7e1b526b03707bbaee3142c692aa0562"
server_key_version: "kuduclusterkey@0"
I20260806 21:14:11.172215 3916 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260806 21:14:11.190891 3942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:11.192864 3916 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260806 21:14:11.193033 3916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-0/wal
uuid: "5bbd400049864275a32fb933c1b99eb3"
format_stamp: "Formatted at 2026-08-06 21:14:11 on dist-test-slave-lgr5"
server_key: "2d5361f11b627db39c77afa43b80d51d"
server_key_iv: "7e1b526b03707bbaee3142c692aa0562"
server_key_version: "kuduclusterkey@0"
I20260806 21:14:11.193203 3916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:14:11.220185 3916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:14:11.223969 3916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:14:11.224220 3916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:14:11.224963 3916 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:14:11.225914 3916 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:14:11.225978 3916 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:11.226039 3916 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:14:11.226068 3916 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:11.236691 3916 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.1:44923
I20260806 21:14:11.238134 3916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
I20260806 21:14:11.239193 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 3916
I20260806 21:14:11.239319 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance
I20260806 21:14:11.239677 17096 external_mini_cluster.cc:1428] Setting key 07794bdb31485799b65d858e11aaff37
I20260806 21:14:11.247364 4120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.1:44923 every 8 connection(s)
WARNING: no policy specified for kudu/127.16.178.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.2@KRBTEST.COM" created.
Aug 06 21:14:11 dist-test-slave-lgr5 krb5kdc[2994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050851, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:14:11.273671 4126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:11.246995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:50643 (local address 127.16.178.62:35317)
0806 21:14:11.249634 (+ 2639us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:11.249639 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:11.252168 (+ 2529us) server_negotiation.cc:407] Connection header received
0806 21:14:11.252247 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:11.252250 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:11.252328 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:11.252429 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:14:11.258959 (+ 6530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:11.259865 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:11.261905 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:11.262255 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:11.262345 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:11.268490 (+ 6145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:14:11.268495 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:14:11.268498 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:14:11.268536 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:14:11.270553 (+ 2017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:11.271326 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:11.271330 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:11.271332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:11.271457 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:11.271956 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:11.271960 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:11.271962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:11.272275 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:14:11.272377 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:11.273276 (+ 899us) server_negotiation.cc:299] Negotiation successful
0806 21:14:11.273441 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2517,"thread_start_us":121,"threads_started":1}
I20260806 21:14:11.275899 4125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:11.245639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35317 (local address 127.16.178.1:50643)
0806 21:14:11.250025 (+ 4386us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:11.250106 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0806 21:14:11.251388 (+ 1282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:14:11.257239 (+ 5851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:14:11.257256 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:14:11.257740 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:14:11.258670 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:11.258704 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:11.260135 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:11.260141 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:11.261712 (+ 1571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:11.261730 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:11.262637 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:11.262642 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:11.262871 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:11.263616 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:14:11.263641 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:14:11.268183 (+ 4542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:14:11.270728 (+ 2545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:11.270736 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:11.270761 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:11.271183 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:11.271596 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:11.271599 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:11.271601 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:11.271829 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:11.272473 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:14:11.272481 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:14:11.272897 (+ 416us) client_negotiation.cc:769] Sending connection context
0806 21:14:11.273156 (+ 259us) client_negotiation.cc:240] Negotiation successful
0806 21:14:11.273592 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4048,"thread_start_us":198,"threads_started":1}
I20260806 21:14:11.277906 4121 heartbeater.cc:347] Connected to a master server at 127.16.178.62:35317
I20260806 21:14:11.278239 4121 heartbeater.cc:464] Registering TS with master...
I20260806 21:14:11.278949 4121 heartbeater.cc:511] Master 127.16.178.62:35317 requested a full tablet report, sending...
I20260806 21:14:11.281095 3785 ts_manager.cc:194] Registered new tserver with Master: 5bbd400049864275a32fb933c1b99eb3 (127.16.178.1:44923)
I20260806 21:14:11.282919 3785 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.1@KRBTEST.COM'} at 127.16.178.1:50643
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab.
Entry for principal kudu/127.16.178.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab.
I20260806 21:14:11.292526 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.178.20:60901/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.2:0
--local_ip_for_outbound_sockets=127.16.178.2
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:35317
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab
--principal=kudu/127.16.178.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:40385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 21:14:11 dist-test-slave-lgr5 krb5kdc[2994](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786050851, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 21:14:11.466467 4128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:14:11.466923 4128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:14:11.467001 4128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:14:11.473145 4128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:14:11.473234 4128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:14:11.473342 4128 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.2
I20260806 21:14:11.481772 4128 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:40385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.16.178.20:60901/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab
--principal=kudu/127.16.178.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:35317
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.2
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:14:11.483412 4128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:14:11.485030 4128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:14:11.499730 4139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:14:11.500192 4136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:14:11.499758 4137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:14:11.501487 4128 server_base.cc:1034] running on GCE node
I20260806 21:14:11.502149 4128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:14:11.503072 4128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:14:11.504314 4128 hybrid_clock.cc:648] HybridClock initialized: now 1786050851504239 us; error 93 us; skew 500 ppm
Aug 06 21:14:11 dist-test-slave-lgr5 krb5kdc[2994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050851, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:14:11.508239 4128 init.cc:377] Logged in from keytab as kudu/127.16.178.2@KRBTEST.COM (short username kudu)
I20260806 21:14:11.509496 4128 webserver.cc:460] Webserver started at http://127.16.178.2:37991/ using document root <none> and password file <none>
I20260806 21:14:11.510264 4128 fs_manager.cc:359] Metadata directory not provided
I20260806 21:14:11.510356 4128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:14:11.510602 4128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 21:14:11 dist-test-slave-lgr5 krb5kdc[2994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050851, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.2@KRBTEST.COM for HTTP/127.16.178.20@KRBTEST.COM
I20260806 21:14:11.530764 4128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-1/data/instance:
uuid: "198a1ec19cf34516ae1005d5197a531d"
format_stamp: "Formatted at 2026-08-06 21:14:11 on dist-test-slave-lgr5"
server_key: "feecf09824a7abcc21c8f8cebeadf263"
server_key_iv: "452a83566e4e307c9dbc0453b15a487e"
server_key_version: "kuduclusterkey@0"
I20260806 21:14:11.531548 4128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance:
uuid: "198a1ec19cf34516ae1005d5197a531d"
format_stamp: "Formatted at 2026-08-06 21:14:11 on dist-test-slave-lgr5"
server_key: "feecf09824a7abcc21c8f8cebeadf263"
server_key_iv: "452a83566e4e307c9dbc0453b15a487e"
server_key_version: "kuduclusterkey@0"
I20260806 21:14:11.537190 4128 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260806 21:14:11.569363 4151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:11.574627 4128 fs_manager.cc:725] Time spent opening block manager: real 0.019s user 0.003s sys 0.001s
I20260806 21:14:11.574824 4128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-1/wal
uuid: "198a1ec19cf34516ae1005d5197a531d"
format_stamp: "Formatted at 2026-08-06 21:14:11 on dist-test-slave-lgr5"
server_key: "feecf09824a7abcc21c8f8cebeadf263"
server_key_iv: "452a83566e4e307c9dbc0453b15a487e"
server_key_version: "kuduclusterkey@0"
I20260806 21:14:11.575006 4128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:14:11.593984 4128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:14:11.597959 4128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:14:11.598172 4128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:14:11.598944 4128 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:14:11.600106 4128 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:14:11.600171 4128 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:11.600230 4128 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:14:11.600257 4128 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:11.611050 4128 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.2:42665
I20260806 21:14:11.611454 4338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.2:42665 every 8 connection(s)
I20260806 21:14:11.612052 4128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
I20260806 21:14:11.614108 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 4128
I20260806 21:14:11.614236 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance
I20260806 21:14:11.614578 17096 external_mini_cluster.cc:1428] Setting key d4c6dab20e8d81e60be2d2e49487d849
Aug 06 21:14:11 dist-test-slave-lgr5 krb5kdc[2994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050851, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.2@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:14:11.626675 4126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:11.614307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:54827 (local address 127.16.178.62:35317)
0806 21:14:11.614493 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:11.614498 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:11.615699 (+ 1201us) server_negotiation.cc:407] Connection header received
0806 21:14:11.616793 (+ 1094us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:11.616796 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:11.616856 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:11.616930 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:14:11.618181 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:11.618723 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:11.619992 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:11.620269 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:11.620375 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:11.623281 (+ 2906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:14:11.623285 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:14:11.623287 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:14:11.623313 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:14:11.624618 (+ 1305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:11.625126 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:11.625129 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:11.625130 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:11.625210 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:11.625565 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:11.625568 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:11.625570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:11.625755 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:14:11.625852 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:11.626364 (+ 512us) server_negotiation.cc:299] Negotiation successful
0806 21:14:11.626502 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 21:14:11.627597 4341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:11.614779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35317 (local address 127.16.178.2:54827)
0806 21:14:11.615538 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:11.615595 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 21:14:11.616560 (+ 965us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:14:11.617126 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:14:11.617134 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:14:11.617428 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:14:11.618021 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:11.618037 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:11.618865 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:11.618868 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:11.619862 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:11.619873 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:11.620429 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:11.620433 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:11.620634 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:11.621310 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:14:11.621365 (+ 55us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:14:11.623100 (+ 1735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:14:11.624731 (+ 1631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:11.624738 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:11.624757 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:11.625018 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:11.625295 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:11.625298 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:11.625300 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:11.625474 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:11.625857 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:14:11.625864 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:14:11.626125 (+ 261us) client_negotiation.cc:769] Sending connection context
0806 21:14:11.626306 (+ 181us) client_negotiation.cc:240] Negotiation successful
0806 21:14:11.626652 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":161,"threads_started":1}
I20260806 21:14:11.628882 4339 heartbeater.cc:347] Connected to a master server at 127.16.178.62:35317
I20260806 21:14:11.629204 4339 heartbeater.cc:464] Registering TS with master...
I20260806 21:14:11.629921 4339 heartbeater.cc:511] Master 127.16.178.62:35317 requested a full tablet report, sending...
I20260806 21:14:11.631253 3785 ts_manager.cc:194] Registered new tserver with Master: 198a1ec19cf34516ae1005d5197a531d (127.16.178.2:42665)
I20260806 21:14:11.632059 3785 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.2@KRBTEST.COM'} at 127.16.178.2:54827
WARNING: no policy specified for kudu/127.16.178.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab.
Entry for principal kudu/127.16.178.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab.
I20260806 21:14:11.671768 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.178.20:60901/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.3:0
--local_ip_for_outbound_sockets=127.16.178.3
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:35317
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab
--principal=kudu/127.16.178.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:40385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 21:14:11 dist-test-slave-lgr5 krb5kdc[2994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050851, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for HTTP/127.16.178.20@KRBTEST.COM
W20260806 21:14:11.843699 4346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:14:11.844170 4346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:14:11.844262 4346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:14:11.849587 4346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:14:11.849661 4346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:14:11.849747 4346 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.3
I20260806 21:14:11.857074 4346 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:40385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.16.178.20:60901/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab
--principal=kudu/127.16.178.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:35317
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4346
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.3
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:14:11.858295 4346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:14:11.859403 4346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:14:11.869436 4357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:14:11.869513 4360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:14:11.869819 4346 server_base.cc:1034] running on GCE node
W20260806 21:14:11.869587 4358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:14:11.870432 4346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:14:11.871243 4346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:14:11.874512 4346 hybrid_clock.cc:648] HybridClock initialized: now 1786050851874467 us; error 48 us; skew 500 ppm
Aug 06 21:14:11 dist-test-slave-lgr5 krb5kdc[2994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050851, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:14:11.877574 4346 init.cc:377] Logged in from keytab as kudu/127.16.178.3@KRBTEST.COM (short username kudu)
I20260806 21:14:11.878564 4346 webserver.cc:460] Webserver started at http://127.16.178.3:38613/ using document root <none> and password file <none>
I20260806 21:14:11.879160 4346 fs_manager.cc:359] Metadata directory not provided
I20260806 21:14:11.879230 4346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:14:11.879480 4346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 21:14:11 dist-test-slave-lgr5 krb5kdc[2994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050851, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.3@KRBTEST.COM for HTTP/127.16.178.20@KRBTEST.COM
I20260806 21:14:11.896543 4346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-2/data/instance:
uuid: "cc0e9a173a0e43f9964b06847c000a4c"
format_stamp: "Formatted at 2026-08-06 21:14:11 on dist-test-slave-lgr5"
server_key: "63e86450f913abda837260fb4a45f5a0"
server_key_iv: "9a7affe89a0c3638af621b4673b207fd"
server_key_version: "kuduclusterkey@0"
I20260806 21:14:11.897562 4346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance:
uuid: "cc0e9a173a0e43f9964b06847c000a4c"
format_stamp: "Formatted at 2026-08-06 21:14:11 on dist-test-slave-lgr5"
server_key: "63e86450f913abda837260fb4a45f5a0"
server_key_iv: "9a7affe89a0c3638af621b4673b207fd"
server_key_version: "kuduclusterkey@0"
I20260806 21:14:11.907796 4346 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.004s sys 0.000s
I20260806 21:14:11.932487 4370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:11.933992 4346 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.004s sys 0.000s
I20260806 21:14:11.934139 4346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-2/wal
uuid: "cc0e9a173a0e43f9964b06847c000a4c"
format_stamp: "Formatted at 2026-08-06 21:14:11 on dist-test-slave-lgr5"
server_key: "63e86450f913abda837260fb4a45f5a0"
server_key_iv: "9a7affe89a0c3638af621b4673b207fd"
server_key_version: "kuduclusterkey@0"
I20260806 21:14:11.934275 4346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:14:11.996914 4346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:14:12.000982 4346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:14:12.001282 4346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:14:12.002187 4346 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:14:12.003546 4346 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:14:12.003667 4346 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260806 21:14:12.003772 4346 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:14:12.003844 4346 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:12.016263 4346 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.3:33207
I20260806 21:14:12.017838 4346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
I20260806 21:14:12.018748 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 4346
I20260806 21:14:12.018903 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance
I20260806 21:14:12.019220 17096 external_mini_cluster.cc:1428] Setting key 49c24e7ad33981f0a9584ad1606fdf8a
I20260806 21:14:12.019222 4562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.3:33207 every 8 connection(s)
Aug 06 21:14:12 dist-test-slave-lgr5 krb5kdc[2994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050851, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.3@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:14:12.038246 4126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:12.022661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:50243 (local address 127.16.178.62:35317)
0806 21:14:12.022820 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:12.022825 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:12.023905 (+ 1080us) server_negotiation.cc:407] Connection header received
0806 21:14:12.025209 (+ 1304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:12.025213 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:12.025274 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:12.025370 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:14:12.026892 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:12.027696 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:12.029504 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:12.029861 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:12.029988 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:12.033180 (+ 3192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:14:12.033185 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:14:12.033187 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:14:12.033220 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:14:12.034999 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:12.035830 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:12.035834 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:12.035836 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:12.035925 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:12.036538 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:12.036542 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:12.036544 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:12.036806 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:14:12.036894 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:12.037903 (+ 1009us) server_negotiation.cc:299] Negotiation successful
0806 21:14:12.038064 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 21:14:12.039398 4566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:12.023101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35317 (local address 127.16.178.3:50243)
0806 21:14:12.023700 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:12.023765 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 21:14:12.024918 (+ 1153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:14:12.025596 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:14:12.025606 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:14:12.026003 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:14:12.026722 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:12.026742 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:12.027870 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:12.027875 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:12.029336 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:12.029351 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:12.030054 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:12.030058 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:12.030271 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:12.031004 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:14:12.031026 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:14:12.032929 (+ 1903us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:14:12.035232 (+ 2303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:12.035240 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:12.035261 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:12.035697 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:12.036046 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:12.036049 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:12.036051 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:12.036413 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:12.036989 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:14:12.036997 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:14:12.037549 (+ 552us) client_negotiation.cc:769] Sending connection context
0806 21:14:12.037787 (+ 238us) client_negotiation.cc:240] Negotiation successful
0806 21:14:12.038174 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":158,"threads_started":1}
I20260806 21:14:12.041204 4563 heartbeater.cc:347] Connected to a master server at 127.16.178.62:35317
I20260806 21:14:12.041508 4563 heartbeater.cc:464] Registering TS with master...
I20260806 21:14:12.042174 4563 heartbeater.cc:511] Master 127.16.178.62:35317 requested a full tablet report, sending...
I20260806 21:14:12.043498 3785 ts_manager.cc:194] Registered new tserver with Master: cc0e9a173a0e43f9964b06847c000a4c (127.16.178.3:33207)
I20260806 21:14:12.044332 3785 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.3@KRBTEST.COM'} at 127.16.178.3:50243
I20260806 21:14:12.047065 17096 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 21:14:12.047909 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 3916
I20260806 21:14:12.069536 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 4128
I20260806 21:14:12.112195 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 4346
Aug 06 21:14:12 dist-test-slave-lgr5 krb5kdc[2994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050851, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for HTTP/127.16.178.20@KRBTEST.COM
I20260806 21:14:12.141039 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 3729
I20260806 21:14:12.161206 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:35317
--webserver_interface=127.16.178.62
--webserver_port=44213
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:40385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:35317
--ranger_config_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.178.20:60901/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.16.178.20:60901/kms with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:14:12.321492 4570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:14:12.321822 4570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:14:12.321870 4570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:14:12.326335 4570 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:14:12.326431 4570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:14:12.326452 4570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:14:12.326467 4570 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:14:12.326480 4570 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:14:12.333626 4570 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:40385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.16.178.20:60901/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:35317
--ranger_config_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:35317
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.178.62
--webserver_port=44213
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4570
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:14:12.334932 4570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:14:12.336169 4570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:14:12.345489 4581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:14:12.345650 4570 server_base.cc:1034] running on GCE node
W20260806 21:14:12.345515 4579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:14:12.345784 4585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:14:12.346395 4570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:14:12.347563 4570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:14:12.348790 4570 hybrid_clock.cc:648] HybridClock initialized: now 1786050852348762 us; error 46 us; skew 500 ppm
Aug 06 21:14:12 dist-test-slave-lgr5 krb5kdc[2994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050852, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:14:12.351759 4570 init.cc:377] Logged in from keytab as kudu/127.16.178.62@KRBTEST.COM (short username kudu)
I20260806 21:14:12.352777 4570 webserver.cc:460] Webserver started at http://127.16.178.62:44213/ using document root <none> and password file <none>
I20260806 21:14:12.353338 4570 fs_manager.cc:359] Metadata directory not provided
I20260806 21:14:12.353415 4570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 21:14:12 dist-test-slave-lgr5 krb5kdc[2994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050852, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for HTTP/127.16.178.20@KRBTEST.COM
I20260806 21:14:12.436184 4570 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:14:12.438429 4600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:12.439700 4570 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 21:14:12.439815 4570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "1f1abc66f1744b849522d675d88595bd"
format_stamp: "Formatted at 2026-08-06 21:14:10 on dist-test-slave-lgr5"
server_key: "a1be7fdfccccfaacf5c07756601d7b2d"
server_key_iv: "0c475cf161ebb1ae0c0a5aff00fb3054"
server_key_version: "kuduclusterkey@0"
I20260806 21:14:12.440435 4570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:14:12.456864 4570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:14:12.459792 4570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:14:12.459987 4570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:14:12.468017 4570 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.62:35317
I20260806 21:14:12.468035 4688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.62:35317 every 8 connection(s)
I20260806 21:14:12.469297 4570 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:14:12.473006 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 4570
I20260806 21:14:12.473446 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.178.20:60901/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.1:44923
--local_ip_for_outbound_sockets=127.16.178.1
--tserver_master_addrs=127.16.178.62:35317
--webserver_port=45153
--webserver_interface=127.16.178.1
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab
--principal=kudu/127.16.178.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:40385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.16.178.20:60901/kms with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 21:14:12.475184 4689 sys_catalog.cc:263] Verifying existing consensus state
I20260806 21:14:12.477669 4689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f1abc66f1744b849522d675d88595bd: Bootstrap starting.
I20260806 21:14:12.500250 4689 log.cc:824] T 00000000000000000000000000000000 P 1f1abc66f1744b849522d675d88595bd: Log is configured to *not* fsync() on all Append() calls
I20260806 21:14:12.511035 4689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f1abc66f1744b849522d675d88595bd: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 21:14:12.511575 4689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f1abc66f1744b849522d675d88595bd: Bootstrap complete.
I20260806 21:14:12.517122 4689 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f1abc66f1744b849522d675d88595bd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f1abc66f1744b849522d675d88595bd" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 35317 } }
I20260806 21:14:12.517712 4689 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f1abc66f1744b849522d675d88595bd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f1abc66f1744b849522d675d88595bd, State: Initialized, Role: FOLLOWER
I20260806 21:14:12.518424 4689 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f1abc66f1744b849522d675d88595bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f1abc66f1744b849522d675d88595bd" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 35317 } }
I20260806 21:14:12.518584 4689 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f1abc66f1744b849522d675d88595bd [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:14:12.518724 4689 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f1abc66f1744b849522d675d88595bd [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:14:12.518853 4689 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f1abc66f1744b849522d675d88595bd [term 1 FOLLOWER]: Advancing to term 2
I20260806 21:14:12.521628 4689 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f1abc66f1744b849522d675d88595bd [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f1abc66f1744b849522d675d88595bd" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 35317 } }
I20260806 21:14:12.522030 4689 leader_election.cc:302] T 00000000000000000000000000000000 P 1f1abc66f1744b849522d675d88595bd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f1abc66f1744b849522d675d88595bd; no voters:
I20260806 21:14:12.522420 4689 leader_election.cc:288] T 00000000000000000000000000000000 P 1f1abc66f1744b849522d675d88595bd [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 21:14:12.522730 4694 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f1abc66f1744b849522d675d88595bd [term 2 FOLLOWER]: Leader election won for term 2
I20260806 21:14:12.523113 4694 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f1abc66f1744b849522d675d88595bd [term 2 LEADER]: Becoming Leader. State: Replica: 1f1abc66f1744b849522d675d88595bd, State: Running, Role: LEADER
I20260806 21:14:12.523691 4694 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f1abc66f1744b849522d675d88595bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f1abc66f1744b849522d675d88595bd" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 35317 } }
I20260806 21:14:12.524020 4689 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f1abc66f1744b849522d675d88595bd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:14:12.525036 4694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f1abc66f1744b849522d675d88595bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1f1abc66f1744b849522d675d88595bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f1abc66f1744b849522d675d88595bd" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 35317 } } }
I20260806 21:14:12.525267 4694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f1abc66f1744b849522d675d88595bd [sys.catalog]: This master's current role is: LEADER
I20260806 21:14:12.525720 4694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f1abc66f1744b849522d675d88595bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f1abc66f1744b849522d675d88595bd. Latest consensus state: current_term: 2 leader_uuid: "1f1abc66f1744b849522d675d88595bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f1abc66f1744b849522d675d88595bd" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 35317 } } }
I20260806 21:14:12.525873 4694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f1abc66f1744b849522d675d88595bd [sys.catalog]: This master's current role is: LEADER
I20260806 21:14:12.526216 4710 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:14:12.526546 4689 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 21:14:12.537523 4710 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:14:12.538458 4710 catalog_manager.cc:1259] Loaded cluster ID: b1a3964411e244ecb39ed4706f74722b
I20260806 21:14:12.538571 4710 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:14:12.541574 4710 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:14:12.542482 4710 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1f1abc66f1744b849522d675d88595bd: Loaded TSK: 0
I20260806 21:14:12.544269 4710 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 21:14:12.640125 4692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:14:12.640592 4692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:14:12.640671 4692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:14:12.648178 4692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:14:12.648273 4692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:14:12.648360 4692 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.1
I20260806 21:14:12.656666 4692 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:40385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.16.178.20:60901/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab
--principal=kudu/127.16.178.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.1:44923
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.178.1
--webserver_port=45153
--tserver_master_addrs=127.16.178.62:35317
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.1
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:14:12.658282 4692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:14:12.659857 4692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:14:12.674427 4747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:14:12.674862 4746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:14:12.674983 4692 server_base.cc:1034] running on GCE node
W20260806 21:14:12.675910 4750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:14:12.676568 4692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:14:12.677591 4692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:14:12.678920 4692 hybrid_clock.cc:648] HybridClock initialized: now 1786050852678848 us; error 106 us; skew 500 ppm
Aug 06 21:14:12 dist-test-slave-lgr5 krb5kdc[2994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050852, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:14:12.683025 4692 init.cc:377] Logged in from keytab as kudu/127.16.178.1@KRBTEST.COM (short username kudu)
I20260806 21:14:12.684616 4692 webserver.cc:460] Webserver started at http://127.16.178.1:45153/ using document root <none> and password file <none>
I20260806 21:14:12.685505 4692 fs_manager.cc:359] Metadata directory not provided
I20260806 21:14:12.685703 4692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 21:14:12 dist-test-slave-lgr5 krb5kdc[2994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050852, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for HTTP/127.16.178.20@KRBTEST.COM
I20260806 21:14:12.737192 4692 fs_manager.cc:709] Time spent opening directory manager: real 0.019s user 0.003s sys 0.000s
I20260806 21:14:12.744115 4761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:12.746260 4692 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.004s sys 0.000s
I20260806 21:14:12.746462 4692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-0/wal
uuid: "5bbd400049864275a32fb933c1b99eb3"
format_stamp: "Formatted at 2026-08-06 21:14:11 on dist-test-slave-lgr5"
server_key: "2d5361f11b627db39c77afa43b80d51d"
server_key_iv: "7e1b526b03707bbaee3142c692aa0562"
server_key_version: "kuduclusterkey@0"
I20260806 21:14:12.747344 4692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:14:12.766364 4692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:14:12.769977 4692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:14:12.770234 4692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:14:12.771044 4692 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:14:12.772161 4692 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:14:12.772220 4692 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:12.772282 4692 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:14:12.772312 4692 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:12.784070 4692 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.1:44923
I20260806 21:14:12.784332 4953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.1:44923 every 8 connection(s)
I20260806 21:14:12.785755 4692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
I20260806 21:14:12.795342 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 4692
I20260806 21:14:12.795888 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.178.20:60901/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.2:42665
--local_ip_for_outbound_sockets=127.16.178.2
--tserver_master_addrs=127.16.178.62:35317
--webserver_port=37991
--webserver_interface=127.16.178.2
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab
--principal=kudu/127.16.178.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:40385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.16.178.20:60901/kms with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 21:14:12 dist-test-slave-lgr5 krb5kdc[2994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050852, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:14:12.826333 4957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:12.788937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35317 (local address 127.16.178.1:57709)
0806 21:14:12.793931 (+ 4994us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:12.794018 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0806 21:14:12.795569 (+ 1551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:14:12.797902 (+ 2333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:14:12.797913 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:14:12.798301 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:14:12.799421 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:12.799447 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:12.801541 (+ 2094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:12.801544 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:12.803015 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:12.803029 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:12.806857 (+ 3828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:12.806863 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:12.807124 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:12.816352 (+ 9228us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:14:12.816383 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:14:12.818802 (+ 2419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:14:12.821916 (+ 3114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:12.821924 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:12.821944 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:12.822377 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:12.822885 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:12.822888 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:12.822890 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:12.823108 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:12.824066 (+ 958us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:14:12.824074 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:14:12.824493 (+ 419us) client_negotiation.cc:769] Sending connection context
0806 21:14:12.824716 (+ 223us) client_negotiation.cc:240] Negotiation successful
0806 21:14:12.825117 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4691,"thread_start_us":181,"threads_started":1}
I20260806 21:14:12.829149 4956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:12.788783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:57709 (local address 127.16.178.62:35317)
0806 21:14:12.793478 (+ 4695us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:12.793510 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:12.794215 (+ 705us) server_negotiation.cc:407] Connection header received
0806 21:14:12.796382 (+ 2167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:12.796399 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:12.796911 (+ 512us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:12.797315 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:14:12.799666 (+ 2351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:12.801344 (+ 1678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:12.803184 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:12.803949 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:12.804166 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:12.819203 (+ 15037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:14:12.819212 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:14:12.819237 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:14:12.819283 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:14:12.821687 (+ 2404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:12.822548 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:12.822557 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:12.822563 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:12.822743 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:12.823246 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:12.823249 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:12.823251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:12.823859 (+ 608us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:14:12.823994 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:12.827584 (+ 3590us) server_negotiation.cc:299] Negotiation successful
0806 21:14:12.828006 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3324,"thread_start_us":135,"threads_started":1}
I20260806 21:14:12.832267 4954 heartbeater.cc:347] Connected to a master server at 127.16.178.62:35317
I20260806 21:14:12.832559 4954 heartbeater.cc:464] Registering TS with master...
I20260806 21:14:12.833218 4954 heartbeater.cc:511] Master 127.16.178.62:35317 requested a full tablet report, sending...
W20260806 21:14:12.835685 4954 heartbeater.cc:644] Failed to heartbeat to 127.16.178.62:35317 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 21:14:12.835820 4954 heartbeater.cc:464] Registering TS with master...
I20260806 21:14:12.836035 4954 heartbeater.cc:511] Master 127.16.178.62:35317 requested a full tablet report, sending...
I20260806 21:14:12.836987 4954 heartbeater.cc:464] Registering TS with master...
I20260806 21:14:12.837183 4954 heartbeater.cc:511] Master 127.16.178.62:35317 requested a full tablet report, sending...
W20260806 21:14:12.838006 4954 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 21:14:12.980439 4959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:14:12.980770 4959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:14:12.980816 4959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:14:12.985509 4959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:14:12.985587 4959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:14:12.985654 4959 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.2
I20260806 21:14:12.993892 4959 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:40385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.16.178.20:60901/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab
--principal=kudu/127.16.178.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.2:42665
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.178.2
--webserver_port=37991
--tserver_master_addrs=127.16.178.62:35317
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4959
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.2
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:14:12.995472 4959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:14:12.996681 4959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:14:13.007886 4966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:14:13.007902 4972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:14:13.007903 4967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:14:13.008807 4959 server_base.cc:1034] running on GCE node
I20260806 21:14:13.009262 4959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:14:13.010142 4959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:14:13.011399 4959 hybrid_clock.cc:648] HybridClock initialized: now 1786050853011368 us; error 43 us; skew 500 ppm
Aug 06 21:14:13 dist-test-slave-lgr5 krb5kdc[2994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050853, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:14:13.014566 4959 init.cc:377] Logged in from keytab as kudu/127.16.178.2@KRBTEST.COM (short username kudu)
I20260806 21:14:13.015578 4959 webserver.cc:460] Webserver started at http://127.16.178.2:37991/ using document root <none> and password file <none>
I20260806 21:14:13.016244 4959 fs_manager.cc:359] Metadata directory not provided
I20260806 21:14:13.016350 4959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 21:14:13 dist-test-slave-lgr5 krb5kdc[2994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050853, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.2@KRBTEST.COM for HTTP/127.16.178.20@KRBTEST.COM
I20260806 21:14:13.039609 4959 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 21:14:13.042912 4981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:13.044668 4959 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 21:14:13.044857 4959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-1/wal
uuid: "198a1ec19cf34516ae1005d5197a531d"
format_stamp: "Formatted at 2026-08-06 21:14:11 on dist-test-slave-lgr5"
server_key: "feecf09824a7abcc21c8f8cebeadf263"
server_key_iv: "452a83566e4e307c9dbc0453b15a487e"
server_key_version: "kuduclusterkey@0"
I20260806 21:14:13.045612 4959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:14:13.076699 4959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:14:13.080762 4959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:14:13.081009 4959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:14:13.081840 4959 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:14:13.083065 4959 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:14:13.083138 4959 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:13.083200 4959 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:14:13.083233 4959 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:13.095300 4959 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.2:42665
I20260806 21:14:13.095366 5169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.2:42665 every 8 connection(s)
I20260806 21:14:13.096964 4959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
I20260806 21:14:13.102011 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 4959
I20260806 21:14:13.102578 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.178.20:60901/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.3:33207
--local_ip_for_outbound_sockets=127.16.178.3
--tserver_master_addrs=127.16.178.62:35317
--webserver_port=38613
--webserver_interface=127.16.178.3
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab
--principal=kudu/127.16.178.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:40385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.16.178.20:60901/kms with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 21:14:13 dist-test-slave-lgr5 krb5kdc[2994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050853, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.2@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:14:13.121017 4956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:13.099480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:38063 (local address 127.16.178.62:35317)
0806 21:14:13.099619 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:13.099623 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:13.104911 (+ 5288us) server_negotiation.cc:407] Connection header received
0806 21:14:13.106535 (+ 1624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:13.106540 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:13.106622 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:13.106728 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:14:13.108557 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:13.109366 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:13.111288 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:13.111659 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:13.111790 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:13.115622 (+ 3832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:14:13.115628 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:14:13.115631 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:14:13.115675 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:14:13.117888 (+ 2213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:13.118774 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:13.118779 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:13.118780 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:13.118892 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:13.119401 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:13.119405 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:13.119407 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:13.119690 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:14:13.119784 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:13.120652 (+ 868us) server_negotiation.cc:299] Negotiation successful
0806 21:14:13.120825 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 21:14:13.123167 5173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:13.100137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35317 (local address 127.16.178.2:38063)
0806 21:14:13.104619 (+ 4482us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:13.104741 (+ 122us) client_negotiation.cc:174] Beginning negotiation
0806 21:14:13.106186 (+ 1445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:14:13.106999 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:14:13.107011 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:14:13.107436 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:14:13.108328 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:13.108354 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:13.109556 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:13.109559 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:13.111124 (+ 1565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:13.111137 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:13.112062 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:13.112067 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:13.112352 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:13.113167 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:14:13.113193 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:14:13.115275 (+ 2082us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:14:13.118098 (+ 2823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:13.118108 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:13.118133 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:13.118618 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:13.119031 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:13.119034 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:13.119036 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:13.119273 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:13.119859 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:14:13.119867 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:14:13.120316 (+ 449us) client_negotiation.cc:769] Sending connection context
0806 21:14:13.121561 (+ 1245us) client_negotiation.cc:240] Negotiation successful
0806 21:14:13.121971 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4111,"thread_start_us":172,"threads_started":1}
I20260806 21:14:13.124991 5170 heartbeater.cc:347] Connected to a master server at 127.16.178.62:35317
I20260806 21:14:13.125286 5170 heartbeater.cc:464] Registering TS with master...
I20260806 21:14:13.125934 5170 heartbeater.cc:511] Master 127.16.178.62:35317 requested a full tablet report, sending...
W20260806 21:14:13.127916 5170 heartbeater.cc:644] Failed to heartbeat to 127.16.178.62:35317 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 21:14:13.128046 5170 heartbeater.cc:464] Registering TS with master...
I20260806 21:14:13.128269 5170 heartbeater.cc:511] Master 127.16.178.62:35317 requested a full tablet report, sending...
I20260806 21:14:13.129273 5170 heartbeater.cc:464] Registering TS with master...
I20260806 21:14:13.129468 5170 heartbeater.cc:511] Master 127.16.178.62:35317 requested a full tablet report, sending...
W20260806 21:14:13.130259 5170 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 21:14:13.257710 5175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:14:13.258020 5175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:14:13.258065 5175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:14:13.262413 5175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:14:13.262511 5175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:14:13.262573 5175 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.3
I20260806 21:14:13.268536 5175 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:40385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.16.178.20:60901/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab
--principal=kudu/127.16.178.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.3:33207
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.178.3
--webserver_port=38613
--tserver_master_addrs=127.16.178.62:35317
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5175
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.3
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:14:13.270076 5175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:14:13.271739 5175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:14:13.287955 5183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:14:13.287955 5187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:14:13.288136 5182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:14:13.288584 5175 server_base.cc:1034] running on GCE node
I20260806 21:14:13.289276 5175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:14:13.290256 5175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:14:13.291472 5175 hybrid_clock.cc:648] HybridClock initialized: now 1786050853291443 us; error 58 us; skew 500 ppm
Aug 06 21:14:13 dist-test-slave-lgr5 krb5kdc[2994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050853, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:14:13.295255 5175 init.cc:377] Logged in from keytab as kudu/127.16.178.3@KRBTEST.COM (short username kudu)
I20260806 21:14:13.296622 5175 webserver.cc:460] Webserver started at http://127.16.178.3:38613/ using document root <none> and password file <none>
I20260806 21:14:13.297428 5175 fs_manager.cc:359] Metadata directory not provided
I20260806 21:14:13.297551 5175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 21:14:13 dist-test-slave-lgr5 krb5kdc[2994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050853, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.3@KRBTEST.COM for HTTP/127.16.178.20@KRBTEST.COM
I20260806 21:14:13.320518 5175 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 21:14:13.325166 5202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:13.327212 5175 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260806 21:14:13.327392 5175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-2/wal
uuid: "cc0e9a173a0e43f9964b06847c000a4c"
format_stamp: "Formatted at 2026-08-06 21:14:11 on dist-test-slave-lgr5"
server_key: "63e86450f913abda837260fb4a45f5a0"
server_key_iv: "9a7affe89a0c3638af621b4673b207fd"
server_key_version: "kuduclusterkey@0"
I20260806 21:14:13.328286 5175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:14:13.347083 5175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:14:13.353538 5175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:14:13.353806 5175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:14:13.354785 5175 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:14:13.356212 5175 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:14:13.356290 5175 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:13.356355 5175 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:14:13.356389 5175 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:13.380422 5175 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.3:33207
I20260806 21:14:13.381037 5387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.3:33207 every 8 connection(s)
I20260806 21:14:13.382280 5175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
I20260806 21:14:13.386570 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 5175
I20260806 21:14:13.405930 5394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:13.387566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58222 (local address 127.16.178.62:35317)
0806 21:14:13.390571 (+ 3005us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:13.390577 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:13.393785 (+ 3208us) server_negotiation.cc:407] Connection header received
0806 21:14:13.393853 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:13.393857 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:13.393926 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:13.394038 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:14:13.395776 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:13.396642 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:13.398567 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:13.399069 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:13.399192 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:13.401401 (+ 2209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:14:13.401406 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:14:13.401410 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:14:13.401451 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:14:13.403440 (+ 1989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:13.403947 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:13.403951 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:13.403953 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:13.404130 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:13.404447 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:13.404452 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:13.404454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:13.404769 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:14:13.405038 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:13.405246 (+ 208us) server_negotiation.cc:299] Negotiation successful
0806 21:14:13.405708 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":2894,"thread_start_us":109,"threads_started":1}
Aug 06 21:14:13 dist-test-slave-lgr5 krb5kdc[2994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050853, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.3@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:14:13.412487 4956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:13.385384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:45833 (local address 127.16.178.62:35317)
0806 21:14:13.385553 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:13.385558 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:13.390895 (+ 5337us) server_negotiation.cc:407] Connection header received
0806 21:14:13.392749 (+ 1854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:13.392754 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:13.392828 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:13.392881 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:14:13.394781 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:13.395615 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:13.398516 (+ 2901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:13.399008 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:13.399108 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:13.408116 (+ 9008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:14:13.408121 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:14:13.408124 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:14:13.408153 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:14:13.409498 (+ 1345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:13.410287 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:13.410291 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:13.410293 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:13.410388 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:13.410905 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:13.410909 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:13.410911 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:13.411180 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:14:13.411284 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:13.412145 (+ 861us) server_negotiation.cc:299] Negotiation successful
0806 21:14:13.412296 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":67}
I20260806 21:14:13.413771 5391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:13.386100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35317 (local address 127.16.178.3:45833)
0806 21:14:13.390655 (+ 4555us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:13.390731 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0806 21:14:13.391939 (+ 1208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:14:13.393398 (+ 1459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:14:13.393408 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:14:13.393831 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:14:13.394590 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:13.394611 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:13.396845 (+ 2234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:13.396849 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:13.398371 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:13.398387 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:13.400285 (+ 1898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:13.400292 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:13.400562 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:13.402468 (+ 1906us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:14:13.402512 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:14:13.407860 (+ 5348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:14:13.409649 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:13.409657 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:13.409690 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:13.410157 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:13.410508 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:13.410512 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:13.410514 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:13.410764 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:13.411327 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:14:13.411336 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:14:13.411761 (+ 425us) client_negotiation.cc:769] Sending connection context
0806 21:14:13.412026 (+ 265us) client_negotiation.cc:240] Negotiation successful
0806 21:14:13.412474 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4239,"thread_start_us":195,"threads_started":1}
I20260806 21:14:13.415155 5388 heartbeater.cc:347] Connected to a master server at 127.16.178.62:35317
I20260806 21:14:13.415493 5388 heartbeater.cc:464] Registering TS with master...
I20260806 21:14:13.416245 5388 heartbeater.cc:511] Master 127.16.178.62:35317 requested a full tablet report, sending...
W20260806 21:14:13.418287 5388 heartbeater.cc:644] Failed to heartbeat to 127.16.178.62:35317 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 21:14:13.418455 5388 heartbeater.cc:464] Registering TS with master...
I20260806 21:14:13.418673 5388 heartbeater.cc:511] Master 127.16.178.62:35317 requested a full tablet report, sending...
I20260806 21:14:13.419711 5388 heartbeater.cc:464] Registering TS with master...
I20260806 21:14:13.419915 5388 heartbeater.cc:511] Master 127.16.178.62:35317 requested a full tablet report, sending...
W20260806 21:14:13.420740 5388 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 06 21:14:13 dist-test-slave-lgr5 krb5kdc[2994](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786050853, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:14:13.838235 4954 heartbeater.cc:464] Registering TS with master...
I20260806 21:14:13.838536 4954 heartbeater.cc:511] Master 127.16.178.62:35317 requested a full tablet report, sending...
I20260806 21:14:13.839635 4626 ts_manager.cc:194] Registered new tserver with Master: 5bbd400049864275a32fb933c1b99eb3 (127.16.178.1:44923)
I20260806 21:14:13.840932 4626 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.1@KRBTEST.COM'} at 127.16.178.1:57709
I20260806 21:14:14.130506 5170 heartbeater.cc:464] Registering TS with master...
I20260806 21:14:14.130832 5170 heartbeater.cc:511] Master 127.16.178.62:35317 requested a full tablet report, sending...
I20260806 21:14:14.131651 4626 ts_manager.cc:194] Registered new tserver with Master: 198a1ec19cf34516ae1005d5197a531d (127.16.178.2:42665)
I20260806 21:14:14.132318 4626 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.2@KRBTEST.COM'} at 127.16.178.2:38063
Aug 06 21:14:14 dist-test-slave-lgr5 krb5kdc[2994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050853, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for HTTP/127.16.178.20@KRBTEST.COM
I20260806 21:14:14.421023 5388 heartbeater.cc:464] Registering TS with master...
I20260806 21:14:14.421376 5388 heartbeater.cc:511] Master 127.16.178.62:35317 requested a full tablet report, sending...
I20260806 21:14:14.422361 4626 ts_manager.cc:194] Registered new tserver with Master: cc0e9a173a0e43f9964b06847c000a4c (127.16.178.3:33207)
I20260806 21:14:14.423079 4626 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.3@KRBTEST.COM'} at 127.16.178.3:45833
I20260806 21:14:14.428830 17096 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 21:14:14.439564 5430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:14.430722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58234 (local address 127.16.178.62:35317)
0806 21:14:14.431393 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:14.431397 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:14.431410 (+ 13us) server_negotiation.cc:407] Connection header received
0806 21:14:14.431452 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:14.431455 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:14.431497 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:14.431587 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:14:14.432576 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:14.433161 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:14.434111 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:14.434356 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:14.434462 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:14.435970 (+ 1508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:14:14.435974 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:14:14.435976 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:14:14.436009 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:14:14.437630 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:14.438132 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:14.438135 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:14.438136 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:14.438226 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:14.438518 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:14.438521 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:14.438522 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:14.438742 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:14:14.438883 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:14.439148 (+ 265us) server_negotiation.cc:299] Negotiation successful
0806 21:14:14.439348 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":77,"threads_started":1}
I20260806 21:14:14.443912 4626 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58234:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:14:14.449031 4626 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:14:14.468000 5442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:14.460345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.3:33207 (local address 127.0.0.1:51970)
0806 21:14:14.462824 (+ 2479us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:14.462962 (+ 138us) client_negotiation.cc:174] Beginning negotiation
0806 21:14:14.463120 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:14:14.463911 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:14:14.463918 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:14:14.463941 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:14:14.464149 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:14.464159 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:14.465362 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:14.465365 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:14.466311 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:14.466326 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:14.467419 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:14.467423 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:14.467537 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:14.467575 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 21:14:14.467689 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 21:14:14.467825 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2272,"thread_start_us":234,"threads_started":1}
I20260806 21:14:14.468024 5441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:14.461383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51970 (local address 127.16.178.3:33207)
0806 21:14:14.462367 (+ 984us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:14.462375 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:14.463069 (+ 694us) server_negotiation.cc:407] Connection header received
0806 21:14:14.463279 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:14.463286 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:14.463729 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:14.464355 (+ 626us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:14:14.464496 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:14.465225 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:14.466489 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:14.467302 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:14.467404 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:14.467586 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:14.467719 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 21:14:14.467822 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":861,"thread_start_us":94,"threads_started":1}
I20260806 21:14:14.471619 5274 tablet_service.cc:1467] Processing CreateTablet for tablet d8817858a01e43a4ab3fcab92005821e (DEFAULT_TABLE table=test-table [id=1e48b5cb7e814d2997e2b6d66420efa6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:14:14.472786 5274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8817858a01e43a4ab3fcab92005821e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:14:14.478238 5442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:14.460399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:44923 (local address 127.0.0.1:40020)
0806 21:14:14.468226 (+ 7827us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:14.468241 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:14:14.468316 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:14:14.472831 (+ 4515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:14:14.472834 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:14:14.472859 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:14:14.472973 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:14.472979 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:14.475414 (+ 2435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:14.475418 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:14.476595 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:14.476609 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:14.477789 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:14.477793 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:14.477895 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:14.477929 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 21:14:14.477998 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 21:14:14.478083 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7766,"thread_start_us":198,"threads_started":1}
I20260806 21:14:14.480808 5440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:14.460634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40020 (local address 127.16.178.1:44923)
0806 21:14:14.472333 (+ 11699us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:14.472341 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:14.472369 (+ 28us) server_negotiation.cc:407] Connection header received
0806 21:14:14.472452 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:14.472457 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:14.472663 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:14.472820 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:14:14.473120 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:14.475259 (+ 2139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:14.476780 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:14.477635 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:14.480026 (+ 2391us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:14.480341 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:14.480434 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 21:14:14.480606 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11557,"thread_start_us":69,"threads_started":1}
I20260806 21:14:14.484752 5444 tablet_bootstrap.cc:492] T d8817858a01e43a4ab3fcab92005821e P cc0e9a173a0e43f9964b06847c000a4c: Bootstrap starting.
I20260806 21:14:14.484380 4833 tablet_service.cc:1467] Processing CreateTablet for tablet d8817858a01e43a4ab3fcab92005821e (DEFAULT_TABLE table=test-table [id=1e48b5cb7e814d2997e2b6d66420efa6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:14:14.486091 4833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8817858a01e43a4ab3fcab92005821e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:14:14.487231 5442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:14.462205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.2:42665 (local address 127.0.0.1:55748)
0806 21:14:14.479664 (+ 17459us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:14.479681 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 21:14:14.479772 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:14:14.481613 (+ 1841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:14:14.481617 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:14:14.481639 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:14:14.481732 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:14.481738 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:14.483300 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:14.483303 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:14.485517 (+ 2214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:14.485531 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:14.486793 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:14.486797 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:14.486891 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:14.486923 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 21:14:14.486990 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 21:14:14.487079 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17391,"thread_start_us":101,"threads_started":1}
I20260806 21:14:14.487679 5444 tablet_bootstrap.cc:654] T d8817858a01e43a4ab3fcab92005821e P cc0e9a173a0e43f9964b06847c000a4c: Neither blocks nor log segments found. Creating new log.
I20260806 21:14:14.488066 5443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:14.462624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55748 (local address 127.16.178.2:42665)
0806 21:14:14.463402 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:14.463408 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:14.481025 (+ 17617us) server_negotiation.cc:407] Connection header received
0806 21:14:14.481128 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:14.481134 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:14.481394 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:14.481892 (+ 498us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:14:14.482055 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:14.483156 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:14.485691 (+ 2535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:14.486648 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:14.487556 (+ 908us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:14.487737 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:14.487799 (+ 62us) server_negotiation.cc:299] Negotiation successful
0806 21:14:14.487905 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":677,"thread_start_us":69,"threads_started":1}
I20260806 21:14:14.489606 5444 log.cc:824] T d8817858a01e43a4ab3fcab92005821e P cc0e9a173a0e43f9964b06847c000a4c: Log is configured to *not* fsync() on all Append() calls
I20260806 21:14:14.491730 5032 tablet_service.cc:1467] Processing CreateTablet for tablet d8817858a01e43a4ab3fcab92005821e (DEFAULT_TABLE table=test-table [id=1e48b5cb7e814d2997e2b6d66420efa6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:14:14.493319 5032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8817858a01e43a4ab3fcab92005821e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:14:14.500423 5448 tablet_bootstrap.cc:492] T d8817858a01e43a4ab3fcab92005821e P 5bbd400049864275a32fb933c1b99eb3: Bootstrap starting.
I20260806 21:14:14.503527 5448 tablet_bootstrap.cc:654] T d8817858a01e43a4ab3fcab92005821e P 5bbd400049864275a32fb933c1b99eb3: Neither blocks nor log segments found. Creating new log.
I20260806 21:14:14.504456 5444 tablet_bootstrap.cc:492] T d8817858a01e43a4ab3fcab92005821e P cc0e9a173a0e43f9964b06847c000a4c: No bootstrap required, opened a new log
I20260806 21:14:14.504804 5444 ts_tablet_manager.cc:1397] T d8817858a01e43a4ab3fcab92005821e P cc0e9a173a0e43f9964b06847c000a4c: Time spent bootstrapping tablet: real 0.020s user 0.007s sys 0.000s
I20260806 21:14:14.504977 5450 tablet_bootstrap.cc:492] T d8817858a01e43a4ab3fcab92005821e P 198a1ec19cf34516ae1005d5197a531d: Bootstrap starting.
I20260806 21:14:14.508013 5450 tablet_bootstrap.cc:654] T d8817858a01e43a4ab3fcab92005821e P 198a1ec19cf34516ae1005d5197a531d: Neither blocks nor log segments found. Creating new log.
I20260806 21:14:14.509182 5450 log.cc:824] T d8817858a01e43a4ab3fcab92005821e P 198a1ec19cf34516ae1005d5197a531d: Log is configured to *not* fsync() on all Append() calls
I20260806 21:14:14.509325 5444 raft_consensus.cc:348] T d8817858a01e43a4ab3fcab92005821e P cc0e9a173a0e43f9964b06847c000a4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e9a173a0e43f9964b06847c000a4c" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33207 } } peers { permanent_uuid: "5bbd400049864275a32fb933c1b99eb3" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 44923 } } peers { permanent_uuid: "198a1ec19cf34516ae1005d5197a531d" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 42665 } }
I20260806 21:14:14.510681 5448 log.cc:824] T d8817858a01e43a4ab3fcab92005821e P 5bbd400049864275a32fb933c1b99eb3: Log is configured to *not* fsync() on all Append() calls
I20260806 21:14:14.511274 5444 raft_consensus.cc:374] T d8817858a01e43a4ab3fcab92005821e P cc0e9a173a0e43f9964b06847c000a4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:14:14.511411 5444 raft_consensus.cc:729] T d8817858a01e43a4ab3fcab92005821e P cc0e9a173a0e43f9964b06847c000a4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc0e9a173a0e43f9964b06847c000a4c, State: Initialized, Role: FOLLOWER
I20260806 21:14:14.512192 5444 consensus_queue.cc:260] T d8817858a01e43a4ab3fcab92005821e P cc0e9a173a0e43f9964b06847c000a4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e9a173a0e43f9964b06847c000a4c" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33207 } } peers { permanent_uuid: "5bbd400049864275a32fb933c1b99eb3" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 44923 } } peers { permanent_uuid: "198a1ec19cf34516ae1005d5197a531d" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 42665 } }
I20260806 21:14:14.512358 5450 tablet_bootstrap.cc:492] T d8817858a01e43a4ab3fcab92005821e P 198a1ec19cf34516ae1005d5197a531d: No bootstrap required, opened a new log
I20260806 21:14:14.512589 5450 ts_tablet_manager.cc:1397] T d8817858a01e43a4ab3fcab92005821e P 198a1ec19cf34516ae1005d5197a531d: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.007s
I20260806 21:14:14.513584 5444 ts_tablet_manager.cc:1428] T d8817858a01e43a4ab3fcab92005821e P cc0e9a173a0e43f9964b06847c000a4c: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20260806 21:14:14.513984 5388 heartbeater.cc:503] Master 127.16.178.62:35317 was elected leader, sending a full tablet report...
I20260806 21:14:14.516824 5450 raft_consensus.cc:348] T d8817858a01e43a4ab3fcab92005821e P 198a1ec19cf34516ae1005d5197a531d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e9a173a0e43f9964b06847c000a4c" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33207 } } peers { permanent_uuid: "5bbd400049864275a32fb933c1b99eb3" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 44923 } } peers { permanent_uuid: "198a1ec19cf34516ae1005d5197a531d" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 42665 } }
I20260806 21:14:14.517280 5450 raft_consensus.cc:374] T d8817858a01e43a4ab3fcab92005821e P 198a1ec19cf34516ae1005d5197a531d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:14:14.517419 5450 raft_consensus.cc:729] T d8817858a01e43a4ab3fcab92005821e P 198a1ec19cf34516ae1005d5197a531d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 198a1ec19cf34516ae1005d5197a531d, State: Initialized, Role: FOLLOWER
I20260806 21:14:14.518044 5450 consensus_queue.cc:260] T d8817858a01e43a4ab3fcab92005821e P 198a1ec19cf34516ae1005d5197a531d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e9a173a0e43f9964b06847c000a4c" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33207 } } peers { permanent_uuid: "5bbd400049864275a32fb933c1b99eb3" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 44923 } } peers { permanent_uuid: "198a1ec19cf34516ae1005d5197a531d" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 42665 } }
I20260806 21:14:14.519433 5450 ts_tablet_manager.cc:1428] T d8817858a01e43a4ab3fcab92005821e P 198a1ec19cf34516ae1005d5197a531d: Time spent starting tablet: real 0.007s user 0.004s sys 0.002s
I20260806 21:14:14.519923 5170 heartbeater.cc:503] Master 127.16.178.62:35317 was elected leader, sending a full tablet report...
I20260806 21:14:14.523823 5448 tablet_bootstrap.cc:492] T d8817858a01e43a4ab3fcab92005821e P 5bbd400049864275a32fb933c1b99eb3: No bootstrap required, opened a new log
I20260806 21:14:14.524132 5448 ts_tablet_manager.cc:1397] T d8817858a01e43a4ab3fcab92005821e P 5bbd400049864275a32fb933c1b99eb3: Time spent bootstrapping tablet: real 0.024s user 0.007s sys 0.000s
I20260806 21:14:14.528453 5448 raft_consensus.cc:348] T d8817858a01e43a4ab3fcab92005821e P 5bbd400049864275a32fb933c1b99eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e9a173a0e43f9964b06847c000a4c" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33207 } } peers { permanent_uuid: "5bbd400049864275a32fb933c1b99eb3" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 44923 } } peers { permanent_uuid: "198a1ec19cf34516ae1005d5197a531d" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 42665 } }
I20260806 21:14:14.528859 5448 raft_consensus.cc:374] T d8817858a01e43a4ab3fcab92005821e P 5bbd400049864275a32fb933c1b99eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:14:14.528944 5448 raft_consensus.cc:729] T d8817858a01e43a4ab3fcab92005821e P 5bbd400049864275a32fb933c1b99eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bbd400049864275a32fb933c1b99eb3, State: Initialized, Role: FOLLOWER
I20260806 21:14:14.529506 5448 consensus_queue.cc:260] T d8817858a01e43a4ab3fcab92005821e P 5bbd400049864275a32fb933c1b99eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e9a173a0e43f9964b06847c000a4c" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33207 } } peers { permanent_uuid: "5bbd400049864275a32fb933c1b99eb3" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 44923 } } peers { permanent_uuid: "198a1ec19cf34516ae1005d5197a531d" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 42665 } }
I20260806 21:14:14.530936 5448 ts_tablet_manager.cc:1428] T d8817858a01e43a4ab3fcab92005821e P 5bbd400049864275a32fb933c1b99eb3: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260806 21:14:14.531477 4954 heartbeater.cc:503] Master 127.16.178.62:35317 was elected leader, sending a full tablet report...
W20260806 21:14:14.539453 4955 tablet.cc:2369] T d8817858a01e43a4ab3fcab92005821e P 5bbd400049864275a32fb933c1b99eb3: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 21:14:14 dist-test-slave-lgr5 krb5kdc[2994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050853, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for HTTP/127.16.178.20@KRBTEST.COM
W20260806 21:14:14.601964 5172 tablet.cc:2369] T d8817858a01e43a4ab3fcab92005821e P 198a1ec19cf34516ae1005d5197a531d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 21:14:14.640290 5390 tablet.cc:2369] T d8817858a01e43a4ab3fcab92005821e P cc0e9a173a0e43f9964b06847c000a4c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 21:14:14.784122 5458 raft_consensus.cc:482] T d8817858a01e43a4ab3fcab92005821e P 198a1ec19cf34516ae1005d5197a531d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:14:14.784403 5458 raft_consensus.cc:504] T d8817858a01e43a4ab3fcab92005821e P 198a1ec19cf34516ae1005d5197a531d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e9a173a0e43f9964b06847c000a4c" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33207 } } peers { permanent_uuid: "5bbd400049864275a32fb933c1b99eb3" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 44923 } } peers { permanent_uuid: "198a1ec19cf34516ae1005d5197a531d" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 42665 } }
I20260806 21:14:14.785925 5458 leader_election.cc:288] T d8817858a01e43a4ab3fcab92005821e P 198a1ec19cf34516ae1005d5197a531d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc0e9a173a0e43f9964b06847c000a4c (127.16.178.3:33207), 5bbd400049864275a32fb933c1b99eb3 (127.16.178.1:44923)
I20260806 21:14:14.798278 5440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:14.786717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:33151 (local address 127.16.178.1:44923)
0806 21:14:14.786834 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:14.786839 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:14.793625 (+ 6786us) server_negotiation.cc:407] Connection header received
0806 21:14:14.793851 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:14.793854 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:14.793905 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:14.793969 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:14:14.794435 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:14.795243 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:14.796934 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:14.797560 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:14.797658 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:14.797711 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:14.798059 (+ 348us) server_negotiation.cc:299] Negotiation successful
0806 21:14:14.798140 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 21:14:14.799010 5464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:14.786277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:44923 (local address 127.16.178.2:33151)
0806 21:14:14.793500 (+ 7223us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:14.793518 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 21:14:14.793717 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:14:14.794125 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:14:14.794128 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:14:14.794147 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:14:14.794292 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:14.794299 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:14.795418 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:14.795422 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:14.796785 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:14.796799 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:14.797806 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:14.797809 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:14.797906 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:14.797929 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 21:14:14.798743 (+ 814us) client_negotiation.cc:240] Negotiation successful
0806 21:14:14.798825 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7132,"thread_start_us":144,"threads_started":1}
I20260806 21:14:14.799881 4874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8817858a01e43a4ab3fcab92005821e" candidate_uuid: "198a1ec19cf34516ae1005d5197a531d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bbd400049864275a32fb933c1b99eb3" is_pre_election: true
I20260806 21:14:14.800303 4874 raft_consensus.cc:2454] T d8817858a01e43a4ab3fcab92005821e P 5bbd400049864275a32fb933c1b99eb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 198a1ec19cf34516ae1005d5197a531d in term 0.
I20260806 21:14:14.800983 4982 leader_election.cc:302] T d8817858a01e43a4ab3fcab92005821e P 198a1ec19cf34516ae1005d5197a531d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 198a1ec19cf34516ae1005d5197a531d, 5bbd400049864275a32fb933c1b99eb3; no voters:
I20260806 21:14:14.801441 5458 raft_consensus.cc:2781] T d8817858a01e43a4ab3fcab92005821e P 198a1ec19cf34516ae1005d5197a531d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:14:14.801535 5458 raft_consensus.cc:482] T d8817858a01e43a4ab3fcab92005821e P 198a1ec19cf34516ae1005d5197a531d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:14:14.801576 5458 raft_consensus.cc:3037] T d8817858a01e43a4ab3fcab92005821e P 198a1ec19cf34516ae1005d5197a531d [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:14:14.802996 5458 raft_consensus.cc:504] T d8817858a01e43a4ab3fcab92005821e P 198a1ec19cf34516ae1005d5197a531d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e9a173a0e43f9964b06847c000a4c" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33207 } } peers { permanent_uuid: "5bbd400049864275a32fb933c1b99eb3" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 44923 } } peers { permanent_uuid: "198a1ec19cf34516ae1005d5197a531d" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 42665 } }
I20260806 21:14:14.803443 5458 leader_election.cc:288] T d8817858a01e43a4ab3fcab92005821e P 198a1ec19cf34516ae1005d5197a531d [CANDIDATE]: Term 1 election: Requested vote from peers cc0e9a173a0e43f9964b06847c000a4c (127.16.178.3:33207), 5bbd400049864275a32fb933c1b99eb3 (127.16.178.1:44923)
I20260806 21:14:14.804329 4874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8817858a01e43a4ab3fcab92005821e" candidate_uuid: "198a1ec19cf34516ae1005d5197a531d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bbd400049864275a32fb933c1b99eb3"
I20260806 21:14:14.804481 4874 raft_consensus.cc:3037] T d8817858a01e43a4ab3fcab92005821e P 5bbd400049864275a32fb933c1b99eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:14:14.805792 4874 raft_consensus.cc:2454] T d8817858a01e43a4ab3fcab92005821e P 5bbd400049864275a32fb933c1b99eb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 198a1ec19cf34516ae1005d5197a531d in term 1.
I20260806 21:14:14.805843 5441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:14.790328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:36039 (local address 127.16.178.3:33207)
0806 21:14:14.790686 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:14.790691 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:14.796368 (+ 5677us) server_negotiation.cc:407] Connection header received
0806 21:14:14.796586 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:14.796589 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:14.796642 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:14.796689 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:14:14.797115 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:14.797887 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:14.799464 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:14.800069 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:14.800201 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:14.800250 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:14.805582 (+ 5332us) server_negotiation.cc:299] Negotiation successful
0806 21:14:14.805702 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265}
I20260806 21:14:14.806299 5466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:14.790233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.3:33207 (local address 127.16.178.2:36039)
0806 21:14:14.796258 (+ 6025us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:14.796275 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 21:14:14.796451 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:14:14.796833 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:14:14.796836 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:14:14.796865 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:14:14.796982 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:14.796988 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:14.798034 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:14.798038 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:14.799315 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:14.799328 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:14.800348 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:14.800351 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:14.800447 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:14.800467 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 21:14:14.806036 (+ 5569us) client_negotiation.cc:240] Negotiation successful
0806 21:14:14.806145 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5947,"thread_start_us":431,"threads_started":1}
I20260806 21:14:14.806345 4982 leader_election.cc:302] T d8817858a01e43a4ab3fcab92005821e P 198a1ec19cf34516ae1005d5197a531d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 198a1ec19cf34516ae1005d5197a531d, 5bbd400049864275a32fb933c1b99eb3; no voters:
I20260806 21:14:14.806826 5458 raft_consensus.cc:2781] T d8817858a01e43a4ab3fcab92005821e P 198a1ec19cf34516ae1005d5197a531d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:14:14.807071 5458 raft_consensus.cc:686] T d8817858a01e43a4ab3fcab92005821e P 198a1ec19cf34516ae1005d5197a531d [term 1 LEADER]: Becoming Leader. State: Replica: 198a1ec19cf34516ae1005d5197a531d, State: Running, Role: LEADER
I20260806 21:14:14.807312 5277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8817858a01e43a4ab3fcab92005821e" candidate_uuid: "198a1ec19cf34516ae1005d5197a531d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc0e9a173a0e43f9964b06847c000a4c" is_pre_election: true
I20260806 21:14:14.807462 5458 consensus_queue.cc:237] T d8817858a01e43a4ab3fcab92005821e P 198a1ec19cf34516ae1005d5197a531d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e9a173a0e43f9964b06847c000a4c" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33207 } } peers { permanent_uuid: "5bbd400049864275a32fb933c1b99eb3" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 44923 } } peers { permanent_uuid: "198a1ec19cf34516ae1005d5197a531d" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 42665 } }
I20260806 21:14:14.807628 5277 raft_consensus.cc:2454] T d8817858a01e43a4ab3fcab92005821e P cc0e9a173a0e43f9964b06847c000a4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 198a1ec19cf34516ae1005d5197a531d in term 0.
I20260806 21:14:14.807974 5288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8817858a01e43a4ab3fcab92005821e" candidate_uuid: "198a1ec19cf34516ae1005d5197a531d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc0e9a173a0e43f9964b06847c000a4c"
I20260806 21:14:14.808108 5288 raft_consensus.cc:3037] T d8817858a01e43a4ab3fcab92005821e P cc0e9a173a0e43f9964b06847c000a4c [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:14:14.809401 5288 raft_consensus.cc:2454] T d8817858a01e43a4ab3fcab92005821e P cc0e9a173a0e43f9964b06847c000a4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 198a1ec19cf34516ae1005d5197a531d in term 1.
I20260806 21:14:14.811735 4626 catalog_manager.cc:5495] T d8817858a01e43a4ab3fcab92005821e P 198a1ec19cf34516ae1005d5197a531d reported cstate change: term changed from 0 to 1, leader changed from <none> to 198a1ec19cf34516ae1005d5197a531d (127.16.178.2). New cstate: current_term: 1 leader_uuid: "198a1ec19cf34516ae1005d5197a531d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0e9a173a0e43f9964b06847c000a4c" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5bbd400049864275a32fb933c1b99eb3" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 44923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "198a1ec19cf34516ae1005d5197a531d" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 42665 } health_report { overall_health: HEALTHY } } }
I20260806 21:14:14.836042 5443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:14.832346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55754 (local address 127.16.178.2:42665)
0806 21:14:14.832485 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:14.832489 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:14.832504 (+ 15us) server_negotiation.cc:407] Connection header received
0806 21:14:14.832665 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:14.832668 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:14.832726 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:14.832819 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:14:14.833073 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:14.833585 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:14.834495 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:14.834766 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:14.834872 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:14.835199 (+ 327us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:14:14.835635 (+ 436us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:14:14.835746 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:14.835817 (+ 71us) server_negotiation.cc:299] Negotiation successful
0806 21:14:14.835883 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 21:14:14.842653 4874 raft_consensus.cc:1261] T d8817858a01e43a4ab3fcab92005821e P 5bbd400049864275a32fb933c1b99eb3 [term 1 FOLLOWER]: Refusing update from remote peer 198a1ec19cf34516ae1005d5197a531d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:14:14.842694 5288 raft_consensus.cc:1261] T d8817858a01e43a4ab3fcab92005821e P cc0e9a173a0e43f9964b06847c000a4c [term 1 FOLLOWER]: Refusing update from remote peer 198a1ec19cf34516ae1005d5197a531d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:14:14.843341 5458 consensus_queue.cc:1035] T d8817858a01e43a4ab3fcab92005821e P 198a1ec19cf34516ae1005d5197a531d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5bbd400049864275a32fb933c1b99eb3" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 44923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:14:14.843524 5467 consensus_queue.cc:1035] T d8817858a01e43a4ab3fcab92005821e P 198a1ec19cf34516ae1005d5197a531d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc0e9a173a0e43f9964b06847c000a4c" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:14:14.852662 5484 mvcc.cc:205] Tried to move back new op lower bound from 7315664301427478528 to 7315664301296345088. Current Snapshot: MvccSnapshot[applied={T|T < 7315664301427478528}]
I20260806 21:14:14.855875 5483 mvcc.cc:205] Tried to move back new op lower bound from 7315664301427478528 to 7315664301296345088. Current Snapshot: MvccSnapshot[applied={T|T < 7315664301427478528}]
I20260806 21:14:14.862443 4627 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:14:14.862605 4627 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:14:14.865170 4627 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1f1abc66f1744b849522d675d88595bd: Sending DeleteTablet for 3 replicas of tablet d8817858a01e43a4ab3fcab92005821e
I20260806 21:14:14.866207 5274 tablet_service.cc:1514] Processing DeleteTablet for tablet d8817858a01e43a4ab3fcab92005821e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:14:14 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:51970
I20260806 21:14:14.866230 4833 tablet_service.cc:1514] Processing DeleteTablet for tablet d8817858a01e43a4ab3fcab92005821e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:14:14 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:40020
I20260806 21:14:14.866726 5499 tablet_replica.cc:330] T d8817858a01e43a4ab3fcab92005821e P 5bbd400049864275a32fb933c1b99eb3: stopping tablet replica
I20260806 21:14:14.867393 5032 tablet_service.cc:1514] Processing DeleteTablet for tablet d8817858a01e43a4ab3fcab92005821e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:14:14 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:55748
I20260806 21:14:14.867691 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 4692
I20260806 21:14:14.884922 5500 tablet_replica.cc:330] T d8817858a01e43a4ab3fcab92005821e P 198a1ec19cf34516ae1005d5197a531d: stopping tablet replica
I20260806 21:14:14.885558 5500 raft_consensus.cc:2229] T d8817858a01e43a4ab3fcab92005821e P 198a1ec19cf34516ae1005d5197a531d [term 1 LEADER]: Raft consensus shutting down.
I20260806 21:14:14.886219 5500 raft_consensus.cc:2258] T d8817858a01e43a4ab3fcab92005821e P 198a1ec19cf34516ae1005d5197a531d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:14:14.886549 5498 tablet_replica.cc:330] T d8817858a01e43a4ab3fcab92005821e P cc0e9a173a0e43f9964b06847c000a4c: stopping tablet replica
I20260806 21:14:14.886934 5498 raft_consensus.cc:2229] T d8817858a01e43a4ab3fcab92005821e P cc0e9a173a0e43f9964b06847c000a4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:14:14.887508 5498 raft_consensus.cc:2258] T d8817858a01e43a4ab3fcab92005821e P cc0e9a173a0e43f9964b06847c000a4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:14:14.889431 5500 ts_tablet_manager.cc:1905] T d8817858a01e43a4ab3fcab92005821e P 198a1ec19cf34516ae1005d5197a531d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 21:14:14.892448 4603 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.178.1:44923 (error 108)
I20260806 21:14:14.892710 5500 ts_tablet_manager.cc:1918] T d8817858a01e43a4ab3fcab92005821e P 198a1ec19cf34516ae1005d5197a531d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:14:14.892802 5500 log.cc:1196] T d8817858a01e43a4ab3fcab92005821e P 198a1ec19cf34516ae1005d5197a531d: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-1/wal/wals/d8817858a01e43a4ab3fcab92005821e
I20260806 21:14:14.893118 5500 ts_tablet_manager.cc:1939] T d8817858a01e43a4ab3fcab92005821e P 198a1ec19cf34516ae1005d5197a531d: Deleting consensus metadata
I20260806 21:14:14.894418 5498 ts_tablet_manager.cc:1905] T d8817858a01e43a4ab3fcab92005821e P cc0e9a173a0e43f9964b06847c000a4c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:14:14.894510 4603 catalog_manager.cc:4841] TS 198a1ec19cf34516ae1005d5197a531d (127.16.178.2:42665): tablet d8817858a01e43a4ab3fcab92005821e (table test-table [id=1e48b5cb7e814d2997e2b6d66420efa6]) successfully deleted
I20260806 21:14:14.894706 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 4959
I20260806 21:14:14.895692 5442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:14.895378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:44923 (local address 127.0.0.1:40030)
0806 21:14:14.895521 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:14.895595 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.1:44923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 21:14:14.902632 4603 catalog_manager.cc:4584] TS 5bbd400049864275a32fb933c1b99eb3 (127.16.178.1:44923): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d8817858a01e43a4ab3fcab92005821e: Network error: Client connection negotiation failed: client connection to 127.16.178.1:44923: connect: Connection refused (error 111)
I20260806 21:14:14.907863 5498 ts_tablet_manager.cc:1918] T d8817858a01e43a4ab3fcab92005821e P cc0e9a173a0e43f9964b06847c000a4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:14:14.908015 5498 log.cc:1196] T d8817858a01e43a4ab3fcab92005821e P cc0e9a173a0e43f9964b06847c000a4c: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786050686797304-17096-0/minicluster-data/ts-2/wal/wals/d8817858a01e43a4ab3fcab92005821e
I20260806 21:14:14.908409 5498 ts_tablet_manager.cc:1939] T d8817858a01e43a4ab3fcab92005821e P cc0e9a173a0e43f9964b06847c000a4c: Deleting consensus metadata
I20260806 21:14:14.910238 4608 catalog_manager.cc:4841] TS cc0e9a173a0e43f9964b06847c000a4c (127.16.178.3:33207): tablet d8817858a01e43a4ab3fcab92005821e (table test-table [id=1e48b5cb7e814d2997e2b6d66420efa6]) successfully deleted
I20260806 21:14:14.910948 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 5175
I20260806 21:14:14.930753 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 4570
I20260806 21:14:14.949971 17096 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 21:14:15.295701 17096 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 21:14:15.295840 17096 mini_ranger.cc:66] Stopping Ranger...
I20260806 21:14:15.641326 17096 mini_ranger.cc:68] Stopped Ranger
2026-08-06 21:14:15.641 UTC [3042] LOG: received smart shutdown request
2026-08-06 21:14:15.645 UTC [3042] LOG: background worker "logical replication launcher" (PID 3053) exited with exit code 1
2026-08-06 21:14:15.646 UTC [3046] LOG: shutting down
2026-08-06 21:14:15.800 UTC [3042] LOG: database system is shut down
2026-08-06T21:14:15Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (51856 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:14:15 dist-test-slave-lgr5 krb5kdc[5521](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:14:15 dist-test-slave-lgr5 krb5kdc[5521](info): set up 2 sockets
Aug 06 21:14:15 dist-test-slave-lgr5 krb5kdc[5521](info): commencing operation
krb5kdc: starting...
W20260806 21:14:17.045836 17096 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Aug 06 21:14:17 dist-test-slave-lgr5 krb5kdc[5521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050857, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:14:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:14:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.178.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
I20260806 21:14:17.178977 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:39619
--webserver_interface=127.16.178.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:36285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:39619,127.16.178.61:39051,127.16.178.60:41617
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:14:17.294180 5545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:14:17.294521 5545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:14:17.294580 5545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:14:17.298867 5545 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:14:17.298959 5545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:14:17.298986 5545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:14:17.299008 5545 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:14:17.299031 5545 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:14:17.304710 5545 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:36285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:39619,127.16.178.61:39051,127.16.178.60:41617
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:39619
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.178.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5545
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:14:17.305893 5545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:14:17.306931 5545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:14:17.315356 5553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:14:17.315366 5557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:14:17.315454 5552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:14:17.315682 5545 server_base.cc:1034] running on GCE node
I20260806 21:14:17.316378 5545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:14:17.317523 5545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:14:17.318696 5545 hybrid_clock.cc:648] HybridClock initialized: now 1786050857318684 us; error 31 us; skew 500 ppm
Aug 06 21:14:17 dist-test-slave-lgr5 krb5kdc[5521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050857, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:14:17.321561 5545 init.cc:377] Logged in from keytab as kudu/127.16.178.62@KRBTEST.COM (short username kudu)
I20260806 21:14:17.322566 5545 webserver.cc:460] Webserver started at http://127.16.178.62:36985/ using document root <none> and password file <none>
I20260806 21:14:17.323104 5545 fs_manager.cc:359] Metadata directory not provided
I20260806 21:14:17.323163 5545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:14:17.323339 5545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:14:17.325502 5545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "131fb1e6b8b5471f9f88f85b18fa6bbf"
format_stamp: "Formatted at 2026-08-06 21:14:17 on dist-test-slave-lgr5"
server_key: "6b5249a9bf816e44b88e1dd3e274badb"
server_key_iv: "6f0543254c06df3de421114d3f8f2061"
server_key_version: "encryptionkey@0"
I20260806 21:14:17.326087 5545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "131fb1e6b8b5471f9f88f85b18fa6bbf"
format_stamp: "Formatted at 2026-08-06 21:14:17 on dist-test-slave-lgr5"
server_key: "6b5249a9bf816e44b88e1dd3e274badb"
server_key_iv: "6f0543254c06df3de421114d3f8f2061"
server_key_version: "encryptionkey@0"
I20260806 21:14:17.330065 5545 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:14:17.332485 5572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:17.333660 5545 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 21:14:17.333772 5545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "131fb1e6b8b5471f9f88f85b18fa6bbf"
format_stamp: "Formatted at 2026-08-06 21:14:17 on dist-test-slave-lgr5"
server_key: "6b5249a9bf816e44b88e1dd3e274badb"
server_key_iv: "6f0543254c06df3de421114d3f8f2061"
server_key_version: "encryptionkey@0"
I20260806 21:14:17.333879 5545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:14:17.345084 5545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:14:17.347815 5545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:14:17.348003 5545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:14:17.355306 5545 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.62:39619
I20260806 21:14:17.355326 5650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.62:39619 every 8 connection(s)
I20260806 21:14:17.356324 5545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:14:17.359122 5652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:14:17.362928 5652 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.178.62" port: 39619 } has no permanent_uuid. Determining permanent_uuid...
I20260806 21:14:17.366385 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 5545
I20260806 21:14:17.366531 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:14:17.366827 17096 external_mini_cluster.cc:1428] Setting key 4178638395ab446e92a437f9c85e90f1
Aug 06 21:14:17 dist-test-slave-lgr5 krb5kdc[5521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050857, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.178.61@KRBTEST.COM; defaulting to no policy
I20260806 21:14:17.378463 5657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:17.366007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52982 (local address 127.16.178.62:39619)
0806 21:14:17.366274 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:17.366282 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:17.366319 (+ 37us) server_negotiation.cc:407] Connection header received
0806 21:14:17.367056 (+ 737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:17.367061 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:17.367239 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:17.367387 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:14:17.368441 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:17.369674 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:17.370836 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:17.371221 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:17.371347 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:17.373916 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:14:17.373922 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:14:17.373932 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:14:17.373953 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:14:17.375575 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:17.376172 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:17.376177 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:17.376178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:17.376287 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:17.376582 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:17.376584 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:17.376586 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:17.376989 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:14:17.377091 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:17.377450 (+ 359us) server_negotiation.cc:299] Negotiation successful
0806 21:14:17.377621 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":86,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.61@KRBTEST.COM" created.
I20260806 21:14:17.378468 5656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:17.365265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:39619 (local address 127.0.0.1:52982)
0806 21:14:17.365750 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:17.365796 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 21:14:17.366736 (+ 940us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:14:17.367453 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:14:17.367459 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:14:17.367899 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:14:17.368294 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:17.368314 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:17.369806 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:17.369809 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:17.370717 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:17.370729 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:17.371378 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:17.371382 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:17.371498 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:17.372176 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:14:17.372192 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:14:17.373768 (+ 1576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:14:17.375691 (+ 1923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:17.375702 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:17.375713 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:17.376026 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:17.376367 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:17.376370 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:17.376371 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:17.376487 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:17.377093 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:14:17.377099 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:14:17.377232 (+ 133us) client_negotiation.cc:769] Sending connection context
0806 21:14:17.377367 (+ 135us) client_negotiation.cc:240] Negotiation successful
0806 21:14:17.377706 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"mutex_wait_us":81,"thread_start_us":112,"threads_started":1}
I20260806 21:14:17.381418 5652 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.178.61" port: 39051 } has no permanent_uuid. Determining permanent_uuid...
I20260806 21:14:17.382519 5656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:17.381969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.61:39051 (local address 127.0.0.1:35664)
0806 21:14:17.382362 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:17.382446 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.61:39051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306}
W20260806 21:14:17.383424 5576 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.178.61:39051: connect: Connection refused (error 111)
I20260806 21:14:17.384343 5656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:17.384098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.61:39051 (local address 127.0.0.1:35666)
0806 21:14:17.384214 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:17.384272 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.61:39051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 21:14:17.384505 5652 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.178.61:39051: Network error: Client connection negotiation failed: client connection to 127.16.178.61:39051: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.61.keytab.
Entry for principal kudu/127.16.178.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.61.keytab.
I20260806 21:14:17.394999 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.61:39051
--webserver_interface=127.16.178.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.61.keytab
--principal=kudu/127.16.178.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:36285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:39619,127.16.178.61:39051,127.16.178.60:41617
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 21:14:17.433779 5652 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.178.61" port: 39051 } attempt: 1
I20260806 21:14:17.434777 5656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:17.434398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.61:39051 (local address 127.0.0.1:35674)
0806 21:14:17.434595 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:17.434677 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.61:39051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260806 21:14:17.435575 5656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:17.435338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.61:39051 (local address 127.0.0.1:35682)
0806 21:14:17.435452 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:17.435508 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.61:39051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 21:14:17.435737 5652 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.178.61:39051: Network error: Client connection negotiation failed: client connection to 127.16.178.61:39051: connect: Connection refused (error 111)
I20260806 21:14:17.503933 5652 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.178.61" port: 39051 } attempt: 2
I20260806 21:14:17.504969 5656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:17.504590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.61:39051 (local address 127.0.0.1:35698)
0806 21:14:17.504782 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:17.504872 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.61:39051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260806 21:14:17.505757 5656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:17.505528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.61:39051 (local address 127.0.0.1:35702)
0806 21:14:17.505632 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:17.505685 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.61:39051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260806 21:14:17.505924 5652 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.178.61:39051: Network error: Client connection negotiation failed: client connection to 127.16.178.61:39051: connect: Connection refused (error 111)
W20260806 21:14:17.516366 5666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:14:17.516657 5666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:14:17.516700 5666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:14:17.520736 5666 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:14:17.520805 5666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:14:17.520828 5666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:14:17.520844 5666 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:14:17.520859 5666 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:14:17.526481 5666 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:36285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:39619,127.16.178.61:39051,127.16.178.60:41617
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.61.keytab
--principal=kudu/127.16.178.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.61:39051
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.16.178.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:14:17.527647 5666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:14:17.528707 5666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:14:17.537662 5673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:14:17.537662 5674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:14:17.537848 5666 server_base.cc:1034] running on GCE node
W20260806 21:14:17.537678 5676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:14:17.538676 5666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:14:17.539848 5666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:14:17.541007 5666 hybrid_clock.cc:648] HybridClock initialized: now 1786050857540979 us; error 42 us; skew 500 ppm
Aug 06 21:14:17 dist-test-slave-lgr5 krb5kdc[5521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050857, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:14:17.543732 5666 init.cc:377] Logged in from keytab as kudu/127.16.178.61@KRBTEST.COM (short username kudu)
I20260806 21:14:17.544677 5666 webserver.cc:460] Webserver started at http://127.16.178.61:43835/ using document root <none> and password file <none>
I20260806 21:14:17.545230 5666 fs_manager.cc:359] Metadata directory not provided
I20260806 21:14:17.545300 5666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:14:17.545523 5666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:14:17.547596 5666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-1/data/instance:
uuid: "0021b3bc18674fc1bd4cdc1509d2e93a"
format_stamp: "Formatted at 2026-08-06 21:14:17 on dist-test-slave-lgr5"
server_key: "f7473e21f19989ebf9152ab8167104f1"
server_key_iv: "5d77b1551de49cfa1b304a3847c57f1e"
server_key_version: "encryptionkey@0"
I20260806 21:14:17.548296 5666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-1/wal/instance:
uuid: "0021b3bc18674fc1bd4cdc1509d2e93a"
format_stamp: "Formatted at 2026-08-06 21:14:17 on dist-test-slave-lgr5"
server_key: "f7473e21f19989ebf9152ab8167104f1"
server_key_iv: "5d77b1551de49cfa1b304a3847c57f1e"
server_key_version: "encryptionkey@0"
I20260806 21:14:17.552116 5666 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 21:14:17.554533 5687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:17.555716 5666 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 21:14:17.555831 5666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-1/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-1/wal
uuid: "0021b3bc18674fc1bd4cdc1509d2e93a"
format_stamp: "Formatted at 2026-08-06 21:14:17 on dist-test-slave-lgr5"
server_key: "f7473e21f19989ebf9152ab8167104f1"
server_key_iv: "5d77b1551de49cfa1b304a3847c57f1e"
server_key_version: "encryptionkey@0"
I20260806 21:14:17.555922 5666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:14:17.577891 5666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:14:17.580965 5666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:14:17.581178 5666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:14:17.588685 5666 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.61:39051
I20260806 21:14:17.588712 5773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.61:39051 every 8 connection(s)
I20260806 21:14:17.589704 5666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-1/data/info.pb
I20260806 21:14:17.592747 5775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:14:17.592823 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 5666
I20260806 21:14:17.592942 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-1/wal/instance
I20260806 21:14:17.593219 17096 external_mini_cluster.cc:1428] Setting key dd6d140bdbb3a3c1d33f00923c5b2edb
I20260806 21:14:17.597080 5652 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.178.61" port: 39051 } attempt: 3
I20260806 21:14:17.597560 5775 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.178.62" port: 39619 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.16.178.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.60@KRBTEST.COM" created.
Aug 06 21:14:17 dist-test-slave-lgr5 krb5kdc[5521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050857, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for kudu/127.16.178.61@KRBTEST.COM
Aug 06 21:14:17 dist-test-slave-lgr5 krb5kdc[5521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050857, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.61@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:14:17.611622 5656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:17.597653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.61:39051 (local address 127.0.0.1:35712)
0806 21:14:17.597886 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:17.597902 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:14:17.598001 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:14:17.600619 (+ 2618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:14:17.600623 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:14:17.600826 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:14:17.600992 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:17.600999 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:17.602923 (+ 1924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:17.602927 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:17.603494 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:17.603502 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:17.604389 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:17.604393 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:17.604484 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:17.604959 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:14:17.604972 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:14:17.607440 (+ 2468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:14:17.609651 (+ 2211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:17.609655 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:17.609657 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:17.610030 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:17.610385 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:17.610389 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:17.610391 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:17.610482 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:17.611108 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:14:17.611111 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:14:17.611207 (+ 96us) client_negotiation.cc:769] Sending connection context
0806 21:14:17.611295 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 21:14:17.611441 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":3}
I20260806 21:14:17.612123 5657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:17.599980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52998 (local address 127.16.178.62:39619)
0806 21:14:17.600153 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:17.600158 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:17.600527 (+ 369us) server_negotiation.cc:407] Connection header received
0806 21:14:17.600752 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:17.600755 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:17.600804 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:17.600881 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:14:17.601959 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:17.602752 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:17.604075 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:17.605199 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:17.605284 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:17.608169 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:14:17.608173 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:14:17.608176 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:14:17.608204 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:14:17.609975 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:17.610422 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:17.610425 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:17.610426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:17.610508 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:17.610799 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:17.610801 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:17.610803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:17.611012 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:14:17.611717 (+ 705us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:17.611826 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 21:14:17.611941 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":49}
I20260806 21:14:17.612936 5782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:17.599873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:39619 (local address 127.0.0.1:52998)
0806 21:14:17.600359 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:17.600419 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 21:14:17.600636 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:14:17.600967 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:14:17.600978 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:14:17.601437 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:14:17.601783 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:17.601803 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:17.602922 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:17.602926 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:17.603953 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:17.603965 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:17.605373 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:17.605378 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:17.605493 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:17.605985 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:14:17.606005 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:14:17.607999 (+ 1994us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:14:17.610088 (+ 2089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:17.610098 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:17.610112 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:17.610340 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:17.610610 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:17.610614 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:17.610616 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:17.610698 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:17.611129 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:14:17.611136 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:14:17.611293 (+ 157us) client_negotiation.cc:769] Sending connection context
0806 21:14:17.611449 (+ 156us) client_negotiation.cc:240] Negotiation successful
0806 21:14:17.611958 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":231,"thread_start_us":166,"threads_started":1}
I20260806 21:14:17.613853 5781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:17.598398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35712 (local address 127.16.178.61:39051)
0806 21:14:17.598870 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:17.598886 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:17.598925 (+ 39us) server_negotiation.cc:407] Connection header received
0806 21:14:17.599702 (+ 777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:17.599729 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:17.600338 (+ 609us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:17.601130 (+ 792us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:14:17.601462 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:17.602844 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:17.603604 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:17.604177 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:17.604343 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:17.607594 (+ 3251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:14:17.607600 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:14:17.607611 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:14:17.607633 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:14:17.609515 (+ 1882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:17.610133 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:17.610139 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:17.610141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:17.610272 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:17.610567 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:17.610570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:17.610571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:17.610989 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:14:17.611110 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:17.611422 (+ 312us) server_negotiation.cc:299] Negotiation successful
0806 21:14:17.613721 (+ 2299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":103,"server-negotiator.queue_time_us":324,"thread_start_us":154,"threads_started":1}
I20260806 21:14:17.614867 5775 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.178.61" port: 39051 } has no permanent_uuid. Determining permanent_uuid...
I20260806 21:14:17.615826 5652 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.178.60" port: 41617 } has no permanent_uuid. Determining permanent_uuid...
I20260806 21:14:17.616526 5656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:17.616273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:41617 (local address 127.0.0.1:47772)
0806 21:14:17.616401 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:17.616459 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.60:41617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 21:14:17.617322 5656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:17.617064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:41617 (local address 127.0.0.1:47774)
0806 21:14:17.617209 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:17.617261 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.60:41617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260806 21:14:17.617477 5652 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.178.60:41617: Network error: Client connection negotiation failed: client connection to 127.16.178.60:41617: connect: Connection refused (error 111)
Aug 06 21:14:17 dist-test-slave-lgr5 krb5kdc[5521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050857, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.61@KRBTEST.COM for kudu/127.16.178.61@KRBTEST.COM
I20260806 21:14:17.622916 5782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:17.615425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.61:39051 (local address 127.0.0.1:35724)
0806 21:14:17.615535 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:17.615549 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:14:17.615628 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:14:17.616055 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:14:17.616058 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:14:17.616227 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:14:17.616322 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:17.616329 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:17.617259 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:17.617263 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:17.617899 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:17.617910 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:17.618519 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:17.618522 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:17.618592 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:17.619075 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:14:17.619086 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:14:17.620245 (+ 1159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:14:17.621656 (+ 1411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:17.621659 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:17.621660 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:17.621837 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:17.622063 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:17.622065 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:17.622066 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:17.622135 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:17.622465 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:14:17.622467 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:14:17.622557 (+ 90us) client_negotiation.cc:769] Sending connection context
0806 21:14:17.622640 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 21:14:17.622764 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 21:14:17.622939 5781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:17.615556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35724 (local address 127.16.178.61:39051)
0806 21:14:17.615851 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:17.615854 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:17.615865 (+ 11us) server_negotiation.cc:407] Connection header received
0806 21:14:17.615892 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:17.615894 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:17.615929 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:17.616036 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:14:17.616447 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:17.617134 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:17.618035 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:17.618411 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:17.618507 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:17.620372 (+ 1865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:14:17.620375 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:14:17.620377 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:14:17.620398 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:14:17.621538 (+ 1140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:17.621919 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:17.621922 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:17.621923 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:17.621991 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:17.622210 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:17.622212 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:17.622213 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:17.622398 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:14:17.622469 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:17.622676 (+ 207us) server_negotiation.cc:299] Negotiation successful
0806 21:14:17.622820 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":35}
I20260806 21:14:17.623620 5775 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.178.60" port: 41617 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.60.keytab.
Entry for principal kudu/127.16.178.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.60.keytab.
I20260806 21:14:17.624298 5782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:17.624042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:41617 (local address 127.0.0.1:47788)
0806 21:14:17.624165 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:17.624245 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.60:41617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260806 21:14:17.624768 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.60:41617
--webserver_interface=127.16.178.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.60.keytab
--principal=kudu/127.16.178.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:36285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:39619,127.16.178.61:39051,127.16.178.60:41617
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:14:17.624883 5694 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.178.60:41617: connect: Connection refused (error 111)
I20260806 21:14:17.625826 5782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:17.625613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:41617 (local address 127.0.0.1:47802)
0806 21:14:17.625727 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:17.625772 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.60:41617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 21:14:17.625950 5775 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.178.60:41617: Network error: Client connection negotiation failed: client connection to 127.16.178.60:41617: connect: Connection refused (error 111)
I20260806 21:14:17.648666 5652 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.178.60" port: 41617 } attempt: 1
I20260806 21:14:17.649554 5656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:17.649218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:41617 (local address 127.0.0.1:47808)
0806 21:14:17.649401 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:17.649464 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.60:41617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260806 21:14:17.650326 5656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:17.650125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:41617 (local address 127.0.0.1:47820)
0806 21:14:17.650224 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:17.650271 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.60:41617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260806 21:14:17.650445 5652 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.178.60:41617: Network error: Client connection negotiation failed: client connection to 127.16.178.60:41617: connect: Connection refused (error 111)
I20260806 21:14:17.675264 5775 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.178.60" port: 41617 } attempt: 1
I20260806 21:14:17.676362 5782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:17.675965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:41617 (local address 127.0.0.1:47826)
0806 21:14:17.676181 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:17.676259 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.60:41617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260806 21:14:17.677130 5782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:17.676883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:41617 (local address 127.0.0.1:47830)
0806 21:14:17.677005 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:17.677060 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.60:41617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 21:14:17.677340 5775 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.178.60:41617: Network error: Client connection negotiation failed: client connection to 127.16.178.60:41617: connect: Connection refused (error 111)
I20260806 21:14:17.725600 5652 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.178.60" port: 41617 } attempt: 2
I20260806 21:14:17.726464 5656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:17.726149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:41617 (local address 127.0.0.1:47834)
0806 21:14:17.726314 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:17.726375 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.60:41617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 21:14:17.727340 5656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:17.727105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:41617 (local address 127.0.0.1:47842)
0806 21:14:17.727221 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:17.727272 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.60:41617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260806 21:14:17.727492 5652 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.178.60:41617: Network error: Client connection negotiation failed: client connection to 127.16.178.60:41617: connect: Connection refused (error 111)
I20260806 21:14:17.745514 5775 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.178.60" port: 41617 } attempt: 2
I20260806 21:14:17.746433 5782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:17.746115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:41617 (local address 127.0.0.1:47844)
0806 21:14:17.746281 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:17.746348 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.60:41617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260806 21:14:17.746744 5787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:14:17.747121 5787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
I20260806 21:14:17.747176 5782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:17.746948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:41617 (local address 127.0.0.1:47854)
0806 21:14:17.747045 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:17.747096 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.60:41617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 21:14:17.747207 5787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:14:17.747398 5775 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.178.60:41617: Network error: Client connection negotiation failed: client connection to 127.16.178.60:41617: connect: Connection refused (error 111)
W20260806 21:14:17.751507 5787 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:14:17.751587 5787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:14:17.751617 5787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:14:17.751653 5787 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:14:17.751688 5787 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:14:17.757359 5787 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:36285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:39619,127.16.178.61:39051,127.16.178.60:41617
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.60.keytab
--principal=kudu/127.16.178.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.60:41617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.16.178.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:14:17.758546 5787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:14:17.759677 5787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:14:17.768635 5796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:14:17.768620 5801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:14:17.768831 5787 server_base.cc:1034] running on GCE node
W20260806 21:14:17.768620 5794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:14:17.769572 5787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:14:17.770738 5787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:14:17.771895 5787 hybrid_clock.cc:648] HybridClock initialized: now 1786050857771875 us; error 32 us; skew 500 ppm
Aug 06 21:14:17 dist-test-slave-lgr5 krb5kdc[5521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050857, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:14:17.775009 5787 init.cc:377] Logged in from keytab as kudu/127.16.178.60@KRBTEST.COM (short username kudu)
I20260806 21:14:17.776114 5787 webserver.cc:460] Webserver started at http://127.16.178.60:34875/ using document root <none> and password file <none>
I20260806 21:14:17.776702 5787 fs_manager.cc:359] Metadata directory not provided
I20260806 21:14:17.776774 5787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:14:17.776994 5787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:14:17.779243 5787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-2/data/instance:
uuid: "8b0a267866c246e4bdd87a7417b60f19"
format_stamp: "Formatted at 2026-08-06 21:14:17 on dist-test-slave-lgr5"
server_key: "3202b16540c26f4198976b69980822d7"
server_key_iv: "1f0472bf15e847d6112e0bdcd7b91e21"
server_key_version: "encryptionkey@0"
I20260806 21:14:17.779784 5787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-2/wal/instance:
uuid: "8b0a267866c246e4bdd87a7417b60f19"
format_stamp: "Formatted at 2026-08-06 21:14:17 on dist-test-slave-lgr5"
server_key: "3202b16540c26f4198976b69980822d7"
server_key_iv: "1f0472bf15e847d6112e0bdcd7b91e21"
server_key_version: "encryptionkey@0"
I20260806 21:14:17.783596 5787 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 21:14:17.786158 5814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:17.787420 5787 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 21:14:17.787551 5787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-2/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-2/wal
uuid: "8b0a267866c246e4bdd87a7417b60f19"
format_stamp: "Formatted at 2026-08-06 21:14:17 on dist-test-slave-lgr5"
server_key: "3202b16540c26f4198976b69980822d7"
server_key_iv: "1f0472bf15e847d6112e0bdcd7b91e21"
server_key_version: "encryptionkey@0"
I20260806 21:14:17.787659 5787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:14:17.816654 5787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:14:17.819985 5787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:14:17.820200 5787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:14:17.826659 5652 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.178.60" port: 41617 } attempt: 3
I20260806 21:14:17.827579 5656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:17.827219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:41617 (local address 127.0.0.1:47864)
0806 21:14:17.827416 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:17.827487 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.60:41617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260806 21:14:17.828126 5787 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.60:41617
I20260806 21:14:17.828179 5896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.60:41617 every 8 connection(s)
I20260806 21:14:17.829424 5787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-2/data/info.pb
I20260806 21:14:17.832432 5899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:14:17.832907 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 5787
I20260806 21:14:17.833042 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786050686797304-17096-0/minicluster-data/master-2/wal/instance
I20260806 21:14:17.833355 17096 external_mini_cluster.cc:1428] Setting key 18289b4f6ae8456bb2bd4143b22208fd
Aug 06 21:14:17 dist-test-slave-lgr5 krb5kdc[5521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050857, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for kudu/127.16.178.60@KRBTEST.COM
I20260806 21:14:17.837338 5899 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.178.62" port: 39619 } has no permanent_uuid. Determining permanent_uuid...
I20260806 21:14:17.838547 5775 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.178.60" port: 41617 } attempt: 3
Aug 06 21:14:17 dist-test-slave-lgr5 krb5kdc[5521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050857, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.178.60@KRBTEST.COM
I20260806 21:14:17.841876 5656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:17.828226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:41617 (local address 127.0.0.1:47872)
0806 21:14:17.828355 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:17.828370 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:14:17.828491 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:14:17.831296 (+ 2805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:14:17.831300 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:14:17.831513 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:14:17.831641 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:17.831650 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:17.833374 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:17.833379 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:17.834152 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:17.834167 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:17.835106 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:17.835111 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:17.835206 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:17.835762 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:14:17.835779 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:14:17.837369 (+ 1590us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:14:17.839849 (+ 2480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:17.839853 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:17.839855 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:17.840066 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:17.840494 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:17.840497 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:17.840499 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:17.840599 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:17.841390 (+ 791us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:14:17.841394 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:14:17.841493 (+ 99us) client_negotiation.cc:769] Sending connection context
0806 21:14:17.841579 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 21:14:17.841709 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 21:14:17.843570 5897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:17.829286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47872 (local address 127.16.178.60:41617)
0806 21:14:17.829792 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:17.829809 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:17.829851 (+ 42us) server_negotiation.cc:407] Connection header received
0806 21:14:17.830586 (+ 735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:17.830605 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:17.830976 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:17.831335 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:14:17.831814 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:17.833195 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:17.834306 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:17.834953 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:17.835146 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:17.837558 (+ 2412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:14:17.837564 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:14:17.837582 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:14:17.837614 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:14:17.839740 (+ 2126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:17.840222 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:17.840228 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:17.840234 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:17.840381 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:17.840713 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:17.840716 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:17.840718 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:17.841273 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:14:17.842128 (+ 855us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:17.842291 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 21:14:17.842590 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":170,"threads_started":1}
I20260806 21:14:17.845049 5903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:17.834306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47878 (local address 127.16.178.60:41617)
0806 21:14:17.834637 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:17.834643 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:17.834654 (+ 11us) server_negotiation.cc:407] Connection header received
0806 21:14:17.834684 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:17.834686 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:17.834720 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:17.834795 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:14:17.835678 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:17.836489 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:17.837443 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:17.837756 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:17.837856 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:17.840651 (+ 2795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:14:17.840655 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:14:17.840657 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:14:17.840683 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:14:17.842316 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:17.842765 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:17.842768 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:17.842770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:17.842868 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:17.843151 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:17.843154 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:17.843155 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:17.843346 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:14:17.844127 (+ 781us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:17.844736 (+ 609us) server_negotiation.cc:299] Negotiation successful
0806 21:14:17.844843 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":139,"threads_started":1}
Aug 06 21:14:17 dist-test-slave-lgr5 krb5kdc[5521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050857, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.61@KRBTEST.COM for kudu/127.16.178.60@KRBTEST.COM
Aug 06 21:14:17 dist-test-slave-lgr5 krb5kdc[5521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050857, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.60@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:14:17.848985 5652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 131fb1e6b8b5471f9f88f85b18fa6bbf: Bootstrap starting.
I20260806 21:14:17.849539 5782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:17.839024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:41617 (local address 127.0.0.1:47886)
0806 21:14:17.839271 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:17.839287 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:14:17.839362 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:14:17.840826 (+ 1464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:14:17.840829 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:14:17.840998 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:14:17.841101 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:17.841109 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:17.842367 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:17.842371 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:17.843139 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:17.843152 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:17.843957 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:17.843960 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:17.844053 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:17.844477 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:14:17.844491 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:14:17.845849 (+ 1358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:14:17.847879 (+ 2030us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:17.847883 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:17.847885 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:17.848150 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:17.848450 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:17.848453 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:17.848455 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:17.848557 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:17.848984 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:14:17.848988 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:14:17.849094 (+ 106us) client_negotiation.cc:769] Sending connection context
0806 21:14:17.849179 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 21:14:17.849311 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165}
I20260806 21:14:17.850135 5907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:17.840072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47886 (local address 127.16.178.60:41617)
0806 21:14:17.840582 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:17.840592 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:17.840604 (+ 12us) server_negotiation.cc:407] Connection header received
0806 21:14:17.840637 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:17.840640 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:17.840681 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:17.841215 (+ 534us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:14:17.841299 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:17.842245 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:17.843288 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:17.843847 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:17.843955 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:17.846004 (+ 2049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:14:17.846008 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:14:17.846011 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:14:17.846033 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:14:17.847749 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:17.848273 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:17.848276 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:17.848278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:17.848354 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:17.848676 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:17.848679 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:17.848681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:17.848870 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:14:17.849797 (+ 927us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:17.849873 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 21:14:17.849973 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":98,"threads_started":1}
I20260806 21:14:17.851935 5652 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 131fb1e6b8b5471f9f88f85b18fa6bbf: Neither blocks nor log segments found. Creating new log.
I20260806 21:14:17.852464 5657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:17.840264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53014 (local address 127.16.178.62:39619)
0806 21:14:17.840387 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:17.840391 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:17.840986 (+ 595us) server_negotiation.cc:407] Connection header received
0806 21:14:17.841191 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:17.841194 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:17.841240 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:17.841293 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:14:17.842344 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:17.843029 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:17.845108 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:17.845404 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:17.845493 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:17.848466 (+ 2973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:14:17.848470 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:14:17.848473 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:14:17.848497 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:14:17.849944 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:17.851077 (+ 1133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:17.851081 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:17.851083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:17.851170 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:17.851451 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:17.851455 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:17.851456 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:17.851674 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:14:17.851753 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:17.852216 (+ 463us) server_negotiation.cc:299] Negotiation successful
0806 21:14:17.852326 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 21:14:17.852934 5906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:17.839368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:39619 (local address 127.0.0.1:53014)
0806 21:14:17.840868 (+ 1500us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:17.840922 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 21:14:17.841077 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:14:17.841460 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:14:17.841466 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:14:17.841908 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:14:17.842215 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:17.842223 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:17.843151 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:17.843154 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:17.844974 (+ 1820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:17.844984 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:17.845637 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:17.845647 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:17.845743 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:17.846205 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:14:17.846219 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:14:17.848324 (+ 2105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:14:17.850649 (+ 2325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:17.850663 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:17.850668 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:17.850943 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:17.851278 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:17.851281 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:17.851283 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:17.851361 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:17.851825 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:14:17.851830 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:14:17.851946 (+ 116us) client_negotiation.cc:769] Sending connection context
0806 21:14:17.852583 (+ 637us) client_negotiation.cc:240] Negotiation successful
0806 21:14:17.852776 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1335,"thread_start_us":103,"threads_started":1}
Aug 06 21:14:17 dist-test-slave-lgr5 krb5kdc[5521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050857, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:14:17.853000 5652 log.cc:824] T 00000000000000000000000000000000 P 131fb1e6b8b5471f9f88f85b18fa6bbf: Log is configured to *not* fsync() on all Append() calls
I20260806 21:14:17.853116 5775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0021b3bc18674fc1bd4cdc1509d2e93a: Bootstrap starting.
I20260806 21:14:17.854403 5899 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.178.61" port: 39051 } has no permanent_uuid. Determining permanent_uuid...
I20260806 21:14:17.855638 5652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 131fb1e6b8b5471f9f88f85b18fa6bbf: No bootstrap required, opened a new log
I20260806 21:14:17.856341 5775 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0021b3bc18674fc1bd4cdc1509d2e93a: Neither blocks nor log segments found. Creating new log.
I20260806 21:14:17.857149 5775 log.cc:824] T 00000000000000000000000000000000 P 0021b3bc18674fc1bd4cdc1509d2e93a: Log is configured to *not* fsync() on all Append() calls
I20260806 21:14:17.859253 5652 raft_consensus.cc:348] T 00000000000000000000000000000000 P 131fb1e6b8b5471f9f88f85b18fa6bbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "131fb1e6b8b5471f9f88f85b18fa6bbf" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 39619 } } peers { permanent_uuid: "0021b3bc18674fc1bd4cdc1509d2e93a" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 39051 } } peers { permanent_uuid: "8b0a267866c246e4bdd87a7417b60f19" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 41617 } }
I20260806 21:14:17.859486 5652 raft_consensus.cc:374] T 00000000000000000000000000000000 P 131fb1e6b8b5471f9f88f85b18fa6bbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:14:17.859572 5652 raft_consensus.cc:729] T 00000000000000000000000000000000 P 131fb1e6b8b5471f9f88f85b18fa6bbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 131fb1e6b8b5471f9f88f85b18fa6bbf, State: Initialized, Role: FOLLOWER
I20260806 21:14:17.859958 5775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0021b3bc18674fc1bd4cdc1509d2e93a: No bootstrap required, opened a new log
I20260806 21:14:17.860098 5908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:17.846855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53024 (local address 127.16.178.62:39619)
0806 21:14:17.847152 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:17.847167 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:17.847180 (+ 13us) server_negotiation.cc:407] Connection header received
0806 21:14:17.847213 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:17.847215 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:17.847263 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:17.847344 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:14:17.848177 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:17.849131 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:17.850417 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:17.850755 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:17.850846 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:17.853531 (+ 2685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:14:17.853535 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:14:17.853537 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:14:17.853561 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:14:17.857759 (+ 4198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:17.858287 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:17.858290 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:17.858292 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:17.858389 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:17.858714 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:17.858717 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:17.858719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:17.858939 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:14:17.859681 (+ 742us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:17.859765 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 21:14:17.859887 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":90,"threads_started":1}
Aug 06 21:14:17 dist-test-slave-lgr5 krb5kdc[5521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050857, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.60@KRBTEST.COM for kudu/127.16.178.61@KRBTEST.COM
I20260806 21:14:17.860414 5652 consensus_queue.cc:260] T 00000000000000000000000000000000 P 131fb1e6b8b5471f9f88f85b18fa6bbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "131fb1e6b8b5471f9f88f85b18fa6bbf" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 39619 } } peers { permanent_uuid: "0021b3bc18674fc1bd4cdc1509d2e93a" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 39051 } } peers { permanent_uuid: "8b0a267866c246e4bdd87a7417b60f19" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 41617 } }
I20260806 21:14:17.862100 5652 sys_catalog.cc:564] T 00000000000000000000000000000000 P 131fb1e6b8b5471f9f88f85b18fa6bbf [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:14:17.864485 5781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:17.855207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35734 (local address 127.16.178.61:39051)
0806 21:14:17.855330 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:17.855335 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:17.855475 (+ 140us) server_negotiation.cc:407] Connection header received
0806 21:14:17.855636 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:17.855639 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:17.855683 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:17.855730 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:14:17.856294 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:17.857016 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:17.858023 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:17.858331 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:17.858402 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:17.860965 (+ 2563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:14:17.860969 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:14:17.860971 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:14:17.861008 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:14:17.862664 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:17.863135 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:17.863139 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:17.863140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:17.863220 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:17.863507 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:17.863510 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:17.863512 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:17.863742 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:14:17.863807 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:17.864215 (+ 408us) server_negotiation.cc:299] Negotiation successful
0806 21:14:17.864329 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 21:14:17.864972 5906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:17.855127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.61:39051 (local address 127.0.0.1:35734)
0806 21:14:17.855414 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:17.855427 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:14:17.855536 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:14:17.855860 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:14:17.855863 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:14:17.856032 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:14:17.856162 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:17.856169 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:17.857152 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:17.857155 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:17.857893 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:17.857903 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:17.858552 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:17.858556 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:17.858637 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:17.859118 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:14:17.859132 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:14:17.860813 (+ 1681us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:14:17.862774 (+ 1961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:17.862777 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:17.862779 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:17.863024 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:17.863321 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:17.863324 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:17.863326 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:17.863407 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:17.863877 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:14:17.863879 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:14:17.863987 (+ 108us) client_negotiation.cc:769] Sending connection context
0806 21:14:17.864675 (+ 688us) client_negotiation.cc:240] Negotiation successful
0806 21:14:17.864819 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223}
I20260806 21:14:17.865656 5899 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.178.60" port: 41617 } has no permanent_uuid. Determining permanent_uuid...
I20260806 21:14:17.866309 5775 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0021b3bc18674fc1bd4cdc1509d2e93a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "131fb1e6b8b5471f9f88f85b18fa6bbf" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 39619 } } peers { permanent_uuid: "0021b3bc18674fc1bd4cdc1509d2e93a" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 39051 } } peers { permanent_uuid: "8b0a267866c246e4bdd87a7417b60f19" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 41617 } }
I20260806 21:14:17.866631 5775 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0021b3bc18674fc1bd4cdc1509d2e93a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:14:17.866739 5775 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0021b3bc18674fc1bd4cdc1509d2e93a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0021b3bc18674fc1bd4cdc1509d2e93a, State: Initialized, Role: FOLLOWER
Aug 06 21:14:17 dist-test-slave-lgr5 krb5kdc[5521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050857, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.178.61@KRBTEST.COM
I20260806 21:14:17.867409 5775 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0021b3bc18674fc1bd4cdc1509d2e93a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "131fb1e6b8b5471f9f88f85b18fa6bbf" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 39619 } } peers { permanent_uuid: "0021b3bc18674fc1bd4cdc1509d2e93a" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 39051 } } peers { permanent_uuid: "8b0a267866c246e4bdd87a7417b60f19" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 41617 } }
I20260806 21:14:17.869796 5775 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0021b3bc18674fc1bd4cdc1509d2e93a [sys.catalog]: configured and running, proceeding with master startup.
Aug 06 21:14:17 dist-test-slave-lgr5 krb5kdc[5521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050857, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.60@KRBTEST.COM for kudu/127.16.178.60@KRBTEST.COM
I20260806 21:14:17.867530 5545 master_runner.cc:186] Error getting master registration for 127.16.178.61:39051: OK, instance_id { permanent_uuid: "0021b3bc18674fc1bd4cdc1509d2e93a" instance_seqno: 1786050857586593 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 21:14:17.871740 5545 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 21:14:17.872371 5917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:17.861495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35742 (local address 127.16.178.61:39051)
0806 21:14:17.861815 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:17.861824 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:17.861836 (+ 12us) server_negotiation.cc:407] Connection header received
0806 21:14:17.861869 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:17.861871 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:17.861918 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:17.862723 (+ 805us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:14:17.862829 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:17.863572 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:17.864604 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:17.864910 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:17.865357 (+ 447us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:17.867880 (+ 2523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:14:17.867884 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:14:17.867887 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:14:17.867913 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:14:17.869672 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:17.870211 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:17.870214 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:17.870216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:17.870305 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:17.870629 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:17.870632 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:17.870634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:17.870864 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:14:17.871660 (+ 796us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:17.871736 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 21:14:17.871863 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":96,"threads_started":1}
I20260806 21:14:17.875396 5907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:17.866722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47900 (local address 127.16.178.60:41617)
0806 21:14:17.866839 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:17.866844 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:17.866855 (+ 11us) server_negotiation.cc:407] Connection header received
0806 21:14:17.866889 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:17.866892 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:17.866932 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:17.867007 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:14:17.867471 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:17.868232 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:17.869215 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:17.869564 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:17.869641 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:17.872007 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:14:17.872011 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:14:17.872013 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:14:17.872033 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:14:17.873749 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:17.874192 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:17.874195 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:17.874196 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:17.874271 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:17.874528 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:17.874531 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:17.874532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:17.874735 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:14:17.874806 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:17.875120 (+ 314us) server_negotiation.cc:299] Negotiation successful
0806 21:14:17.875243 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260806 21:14:17.875528 5934 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 131fb1e6b8b5471f9f88f85b18fa6bbf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 21:14:17.875600 5934 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 21:14:17.875788 5906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:17.866150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:41617 (local address 127.0.0.1:47900)
0806 21:14:17.866418 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:17.866432 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:14:17.866507 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:14:17.867114 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:14:17.867117 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:14:17.867273 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:14:17.867363 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:17.867371 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:17.868336 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:17.868338 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:17.869089 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:17.869102 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:17.869782 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:17.869785 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:17.869858 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:17.870273 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:14:17.870286 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:14:17.871875 (+ 1589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:14:17.873847 (+ 1972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:17.873849 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:17.873851 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:17.874096 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:17.874363 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:17.874366 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:17.874368 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:17.874446 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:17.874858 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:14:17.874861 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:14:17.874940 (+ 79us) client_negotiation.cc:769] Sending connection context
0806 21:14:17.875521 (+ 581us) client_negotiation.cc:240] Negotiation successful
0806 21:14:17.875651 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":80}
I20260806 21:14:17.875999 5916 sys_catalog.cc:455] T 00000000000000000000000000000000 P 131fb1e6b8b5471f9f88f85b18fa6bbf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "131fb1e6b8b5471f9f88f85b18fa6bbf" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 39619 } } peers { permanent_uuid: "0021b3bc18674fc1bd4cdc1509d2e93a" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 39051 } } peers { permanent_uuid: "8b0a267866c246e4bdd87a7417b60f19" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 41617 } } }
I20260806 21:14:17.876147 5916 sys_catalog.cc:458] T 00000000000000000000000000000000 P 131fb1e6b8b5471f9f88f85b18fa6bbf [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:14:17.876403 5936 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0021b3bc18674fc1bd4cdc1509d2e93a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "131fb1e6b8b5471f9f88f85b18fa6bbf" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 39619 } } peers { permanent_uuid: "0021b3bc18674fc1bd4cdc1509d2e93a" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 39051 } } peers { permanent_uuid: "8b0a267866c246e4bdd87a7417b60f19" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 41617 } } }
I20260806 21:14:17.876555 5936 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0021b3bc18674fc1bd4cdc1509d2e93a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:14:17.878576 5899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b0a267866c246e4bdd87a7417b60f19: Bootstrap starting.
W20260806 21:14:17.880122 5954 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0021b3bc18674fc1bd4cdc1509d2e93a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 21:14:17.880219 5954 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 21:14:17.881460 5899 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b0a267866c246e4bdd87a7417b60f19: Neither blocks nor log segments found. Creating new log.
I20260806 21:14:17.882414 5899 log.cc:824] T 00000000000000000000000000000000 P 8b0a267866c246e4bdd87a7417b60f19: Log is configured to *not* fsync() on all Append() calls
I20260806 21:14:17.885069 5899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b0a267866c246e4bdd87a7417b60f19: No bootstrap required, opened a new log
I20260806 21:14:17.890113 5899 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b0a267866c246e4bdd87a7417b60f19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "131fb1e6b8b5471f9f88f85b18fa6bbf" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 39619 } } peers { permanent_uuid: "0021b3bc18674fc1bd4cdc1509d2e93a" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 39051 } } peers { permanent_uuid: "8b0a267866c246e4bdd87a7417b60f19" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 41617 } }
I20260806 21:14:17.890481 5899 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8b0a267866c246e4bdd87a7417b60f19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:14:17.890605 5899 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b0a267866c246e4bdd87a7417b60f19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b0a267866c246e4bdd87a7417b60f19, State: Initialized, Role: FOLLOWER
I20260806 21:14:17.891172 5899 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b0a267866c246e4bdd87a7417b60f19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "131fb1e6b8b5471f9f88f85b18fa6bbf" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 39619 } } peers { permanent_uuid: "0021b3bc18674fc1bd4cdc1509d2e93a" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 39051 } } peers { permanent_uuid: "8b0a267866c246e4bdd87a7417b60f19" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 41617 } }
I20260806 21:14:17.888442 5666 master_runner.cc:186] Error getting master registration for 127.16.178.60:41617: OK, instance_id { permanent_uuid: "8b0a267866c246e4bdd87a7417b60f19" instance_seqno: 1786050857825249 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 21:14:17.891840 5666 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 21:14:17.892041 17096 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:14:17.892940 5899 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b0a267866c246e4bdd87a7417b60f19 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:14:17.896343 5964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b0a267866c246e4bdd87a7417b60f19 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "131fb1e6b8b5471f9f88f85b18fa6bbf" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 39619 } } peers { permanent_uuid: "0021b3bc18674fc1bd4cdc1509d2e93a" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 39051 } } peers { permanent_uuid: "8b0a267866c246e4bdd87a7417b60f19" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 41617 } } }
I20260806 21:14:17.896628 5964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b0a267866c246e4bdd87a7417b60f19 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 21:14:17.900342 5993 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8b0a267866c246e4bdd87a7417b60f19: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 21:14:17.900499 5993 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 21:14:17.901414 5787 master_runner.cc:263] No leader master found from master 8b0a267866c246e4bdd87a7417b60f19
I20260806 21:14:17.901490 5787 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 21:14:17.910779 5917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:17.900381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35750 (local address 127.16.178.61:39051)
0806 21:14:17.900627 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:17.900631 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:17.901882 (+ 1251us) server_negotiation.cc:407] Connection header received
0806 21:14:17.902063 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:17.902067 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:17.902114 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:17.902164 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:14:17.903131 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:17.903925 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:17.905101 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:17.905454 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:17.905543 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:17.907172 (+ 1629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:14:17.907176 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:14:17.907178 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:14:17.907210 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:14:17.908832 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:17.909318 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:17.909322 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:17.909324 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:17.909411 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:17.909723 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:17.909727 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:17.909729 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:17.909959 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:14:17.910077 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:17.910437 (+ 360us) server_negotiation.cc:299] Negotiation successful
0806 21:14:17.910598 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166}
I20260806 21:14:17.918069 5908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:17.894605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53036 (local address 127.16.178.62:39619)
0806 21:14:17.894733 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:17.894738 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:17.896301 (+ 1563us) server_negotiation.cc:407] Connection header received
0806 21:14:17.896504 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:17.896507 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:17.896556 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:17.896606 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:14:17.899150 (+ 2544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:17.899904 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:17.904299 (+ 4395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:17.904727 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:17.904809 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:17.907306 (+ 2497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:14:17.907310 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:14:17.907313 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:14:17.907346 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:14:17.908959 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:17.910554 (+ 1595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:17.910557 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:17.910559 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:17.914920 (+ 4361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:17.916873 (+ 1953us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:17.916877 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:17.916879 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:17.917164 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:14:17.917306 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:17.917728 (+ 422us) server_negotiation.cc:299] Negotiation successful
0806 21:14:17.917890 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 21:14:17.919884 5907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:17.895389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47916 (local address 127.16.178.60:41617)
0806 21:14:17.895508 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:17.895512 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:17.897536 (+ 2024us) server_negotiation.cc:407] Connection header received
0806 21:14:17.898166 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:17.898169 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:17.898212 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:17.898268 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:14:17.901768 (+ 3500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:17.902535 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:17.905354 (+ 2819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:17.905716 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:17.905790 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:17.910188 (+ 4398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:14:17.910192 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:14:17.910195 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:14:17.910227 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:14:17.916480 (+ 6253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:17.917366 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:17.917370 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:17.917371 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:17.917539 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:17.918037 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:17.918041 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:17.918043 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:17.918863 (+ 820us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:14:17.919101 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:17.919538 (+ 437us) server_negotiation.cc:299] Negotiation successful
0806 21:14:17.919711 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 21:14:17.926672 5936 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0021b3bc18674fc1bd4cdc1509d2e93a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:14:17.926946 5936 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0021b3bc18674fc1bd4cdc1509d2e93a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "131fb1e6b8b5471f9f88f85b18fa6bbf" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 39619 } } peers { permanent_uuid: "0021b3bc18674fc1bd4cdc1509d2e93a" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 39051 } } peers { permanent_uuid: "8b0a267866c246e4bdd87a7417b60f19" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 41617 } }
I20260806 21:14:17.928006 5936 leader_election.cc:288] T 00000000000000000000000000000000 P 0021b3bc18674fc1bd4cdc1509d2e93a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 131fb1e6b8b5471f9f88f85b18fa6bbf (127.16.178.62:39619), 8b0a267866c246e4bdd87a7417b60f19 (127.16.178.60:41617)
I20260806 21:14:17.930014 5852 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0021b3bc18674fc1bd4cdc1509d2e93a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b0a267866c246e4bdd87a7417b60f19" is_pre_election: true
I20260806 21:14:17.930395 5852 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8b0a267866c246e4bdd87a7417b60f19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0021b3bc18674fc1bd4cdc1509d2e93a in term 0.
I20260806 21:14:17.930847 5694 leader_election.cc:302] T 00000000000000000000000000000000 P 0021b3bc18674fc1bd4cdc1509d2e93a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0021b3bc18674fc1bd4cdc1509d2e93a, 8b0a267866c246e4bdd87a7417b60f19; no voters:
I20260806 21:14:17.931208 5936 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0021b3bc18674fc1bd4cdc1509d2e93a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:14:17.931313 5936 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0021b3bc18674fc1bd4cdc1509d2e93a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:14:17.931360 5936 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0021b3bc18674fc1bd4cdc1509d2e93a [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:14:17.932929 5936 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0021b3bc18674fc1bd4cdc1509d2e93a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "131fb1e6b8b5471f9f88f85b18fa6bbf" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 39619 } } peers { permanent_uuid: "0021b3bc18674fc1bd4cdc1509d2e93a" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 39051 } } peers { permanent_uuid: "8b0a267866c246e4bdd87a7417b60f19" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 41617 } }
I20260806 21:14:17.933542 5936 leader_election.cc:288] T 00000000000000000000000000000000 P 0021b3bc18674fc1bd4cdc1509d2e93a [CANDIDATE]: Term 1 election: Requested vote from peers 131fb1e6b8b5471f9f88f85b18fa6bbf (127.16.178.62:39619), 8b0a267866c246e4bdd87a7417b60f19 (127.16.178.60:41617)
I20260806 21:14:17.934072 5613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0021b3bc18674fc1bd4cdc1509d2e93a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "131fb1e6b8b5471f9f88f85b18fa6bbf"
I20260806 21:14:17.934413 5613 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 131fb1e6b8b5471f9f88f85b18fa6bbf [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:14:17.935233 5614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0021b3bc18674fc1bd4cdc1509d2e93a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "131fb1e6b8b5471f9f88f85b18fa6bbf" is_pre_election: true
I20260806 21:14:17.935928 5613 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 131fb1e6b8b5471f9f88f85b18fa6bbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0021b3bc18674fc1bd4cdc1509d2e93a in term 1.
I20260806 21:14:17.936339 5695 leader_election.cc:302] T 00000000000000000000000000000000 P 0021b3bc18674fc1bd4cdc1509d2e93a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0021b3bc18674fc1bd4cdc1509d2e93a, 131fb1e6b8b5471f9f88f85b18fa6bbf; no voters:
I20260806 21:14:17.936399 5852 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0021b3bc18674fc1bd4cdc1509d2e93a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b0a267866c246e4bdd87a7417b60f19"
I20260806 21:14:17.936533 5852 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b0a267866c246e4bdd87a7417b60f19 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:14:17.937855 5852 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8b0a267866c246e4bdd87a7417b60f19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0021b3bc18674fc1bd4cdc1509d2e93a in term 1.
I20260806 21:14:17.938148 5936 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0021b3bc18674fc1bd4cdc1509d2e93a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:14:17.938427 5936 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0021b3bc18674fc1bd4cdc1509d2e93a [term 1 LEADER]: Becoming Leader. State: Replica: 0021b3bc18674fc1bd4cdc1509d2e93a, State: Running, Role: LEADER
I20260806 21:14:17.938812 5936 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0021b3bc18674fc1bd4cdc1509d2e93a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "131fb1e6b8b5471f9f88f85b18fa6bbf" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 39619 } } peers { permanent_uuid: "0021b3bc18674fc1bd4cdc1509d2e93a" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 39051 } } peers { permanent_uuid: "8b0a267866c246e4bdd87a7417b60f19" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 41617 } }
I20260806 21:14:17.940841 5936 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0021b3bc18674fc1bd4cdc1509d2e93a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0021b3bc18674fc1bd4cdc1509d2e93a. Latest consensus state: current_term: 1 leader_uuid: "0021b3bc18674fc1bd4cdc1509d2e93a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "131fb1e6b8b5471f9f88f85b18fa6bbf" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 39619 } } peers { permanent_uuid: "0021b3bc18674fc1bd4cdc1509d2e93a" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 39051 } } peers { permanent_uuid: "8b0a267866c246e4bdd87a7417b60f19" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 41617 } } }
I20260806 21:14:17.940956 5936 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0021b3bc18674fc1bd4cdc1509d2e93a [sys.catalog]: This master's current role is: LEADER
I20260806 21:14:17.948328 6002 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:14:17.949311 6002 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:14:17.954758 5852 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8b0a267866c246e4bdd87a7417b60f19 [term 1 FOLLOWER]: Refusing update from remote peer 0021b3bc18674fc1bd4cdc1509d2e93a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:14:17.955379 5936 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0021b3bc18674fc1bd4cdc1509d2e93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b0a267866c246e4bdd87a7417b60f19" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 41617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:14:17.956288 5613 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 131fb1e6b8b5471f9f88f85b18fa6bbf [term 1 FOLLOWER]: Refusing update from remote peer 0021b3bc18674fc1bd4cdc1509d2e93a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:14:17.956773 5936 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0021b3bc18674fc1bd4cdc1509d2e93a [LEADER]: Connected to new peer: Peer: permanent_uuid: "131fb1e6b8b5471f9f88f85b18fa6bbf" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 39619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:14:17.963270 5964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b0a267866c246e4bdd87a7417b60f19 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0021b3bc18674fc1bd4cdc1509d2e93a. Latest consensus state: current_term: 1 leader_uuid: "0021b3bc18674fc1bd4cdc1509d2e93a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "131fb1e6b8b5471f9f88f85b18fa6bbf" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 39619 } } peers { permanent_uuid: "0021b3bc18674fc1bd4cdc1509d2e93a" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 39051 } } peers { permanent_uuid: "8b0a267866c246e4bdd87a7417b60f19" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 41617 } } }
I20260806 21:14:17.963409 5964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b0a267866c246e4bdd87a7417b60f19 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:14:17.964314 5916 sys_catalog.cc:455] T 00000000000000000000000000000000 P 131fb1e6b8b5471f9f88f85b18fa6bbf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0021b3bc18674fc1bd4cdc1509d2e93a. Latest consensus state: current_term: 1 leader_uuid: "0021b3bc18674fc1bd4cdc1509d2e93a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "131fb1e6b8b5471f9f88f85b18fa6bbf" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 39619 } } peers { permanent_uuid: "0021b3bc18674fc1bd4cdc1509d2e93a" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 39051 } } peers { permanent_uuid: "8b0a267866c246e4bdd87a7417b60f19" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 41617 } } }
I20260806 21:14:17.964439 5916 sys_catalog.cc:458] T 00000000000000000000000000000000 P 131fb1e6b8b5471f9f88f85b18fa6bbf [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:14:17.976802 5936 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0021b3bc18674fc1bd4cdc1509d2e93a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0021b3bc18674fc1bd4cdc1509d2e93a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "131fb1e6b8b5471f9f88f85b18fa6bbf" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 39619 } } peers { permanent_uuid: "0021b3bc18674fc1bd4cdc1509d2e93a" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 39051 } } peers { permanent_uuid: "8b0a267866c246e4bdd87a7417b60f19" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 41617 } } }
I20260806 21:14:17.976963 5936 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0021b3bc18674fc1bd4cdc1509d2e93a [sys.catalog]: This master's current role is: LEADER
I20260806 21:14:17.977125 6003 mvcc.cc:205] Tried to move back new op lower bound from 7315664314175160320 to 7315664314121084928. Current Snapshot: MvccSnapshot[applied={T|T < 7315664314175160320}]
I20260806 21:14:17.977265 5964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b0a267866c246e4bdd87a7417b60f19 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0021b3bc18674fc1bd4cdc1509d2e93a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "131fb1e6b8b5471f9f88f85b18fa6bbf" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 39619 } } peers { permanent_uuid: "0021b3bc18674fc1bd4cdc1509d2e93a" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 39051 } } peers { permanent_uuid: "8b0a267866c246e4bdd87a7417b60f19" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 41617 } } }
I20260806 21:14:17.977380 5964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b0a267866c246e4bdd87a7417b60f19 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:14:17.977918 6005 mvcc.cc:205] Tried to move back new op lower bound from 7315664314175160320 to 7315664314121084928. Current Snapshot: MvccSnapshot[applied={T|T < 7315664314175160320}]
I20260806 21:14:17.979161 5936 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0021b3bc18674fc1bd4cdc1509d2e93a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0021b3bc18674fc1bd4cdc1509d2e93a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "131fb1e6b8b5471f9f88f85b18fa6bbf" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 39619 } } peers { permanent_uuid: "0021b3bc18674fc1bd4cdc1509d2e93a" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 39051 } } peers { permanent_uuid: "8b0a267866c246e4bdd87a7417b60f19" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 41617 } } }
I20260806 21:14:17.979305 5936 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0021b3bc18674fc1bd4cdc1509d2e93a [sys.catalog]: This master's current role is: LEADER
I20260806 21:14:17.980695 5916 sys_catalog.cc:455] T 00000000000000000000000000000000 P 131fb1e6b8b5471f9f88f85b18fa6bbf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0021b3bc18674fc1bd4cdc1509d2e93a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "131fb1e6b8b5471f9f88f85b18fa6bbf" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 39619 } } peers { permanent_uuid: "0021b3bc18674fc1bd4cdc1509d2e93a" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 39051 } } peers { permanent_uuid: "8b0a267866c246e4bdd87a7417b60f19" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 41617 } } }
I20260806 21:14:17.980822 5916 sys_catalog.cc:458] T 00000000000000000000000000000000 P 131fb1e6b8b5471f9f88f85b18fa6bbf [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:14:17.990387 6002 catalog_manager.cc:1347] Generated new cluster ID: 879e791986dd48a28f505d087d3540e1
I20260806 21:14:17.990510 6002 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:14:18.019507 6002 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:14:18.020937 6002 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:14:18.049727 6002 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0021b3bc18674fc1bd4cdc1509d2e93a: Generated new TSK 0
I20260806 21:14:18.050727 6002 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:14:18.877172 5934 catalog_manager.cc:1259] Loaded cluster ID: 879e791986dd48a28f505d087d3540e1
I20260806 21:14:18.877276 5934 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 131fb1e6b8b5471f9f88f85b18fa6bbf: loading cluster ID for follower catalog manager: success
I20260806 21:14:18.879057 5934 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 131fb1e6b8b5471f9f88f85b18fa6bbf: acquiring CA information for follower catalog manager: success
I20260806 21:14:18.880445 5934 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 131fb1e6b8b5471f9f88f85b18fa6bbf: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 21:14:18.902034 5993 catalog_manager.cc:1259] Loaded cluster ID: 879e791986dd48a28f505d087d3540e1
I20260806 21:14:18.902125 5993 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8b0a267866c246e4bdd87a7417b60f19: loading cluster ID for follower catalog manager: success
I20260806 21:14:18.903916 5993 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8b0a267866c246e4bdd87a7417b60f19: acquiring CA information for follower catalog manager: success
I20260806 21:14:18.905169 5993 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8b0a267866c246e4bdd87a7417b60f19: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 21:14:19.129392 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 5545
I20260806 21:14:19.149721 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 5666
I20260806 21:14:19.163936 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 5787
2026-08-06T21:14:19Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3275 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050686797304-17096-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:14:19 dist-test-slave-lgr5 krb5kdc[6026](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:14:19 dist-test-slave-lgr5 krb5kdc[6026](info): set up 2 sockets
Aug 06 21:14:19 dist-test-slave-lgr5 krb5kdc[6026](info): commencing operation
krb5kdc: starting...
W20260806 21:14:20.325505 17096 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Aug 06 21:14:20 dist-test-slave-lgr5 krb5kdc[6026](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:14:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:14:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.178.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
I20260806 21:14:20.453580 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:38989
--webserver_interface=127.16.178.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:45227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:38989
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:14:20.569162 6044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:14:20.569464 6044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:14:20.569514 6044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:14:20.573580 6044 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:14:20.573655 6044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:14:20.573673 6044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:14:20.573689 6044 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:14:20.573702 6044 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:14:20.579360 6044 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:45227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:38989
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:38989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.178.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:14:20.580611 6044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:14:20.581686 6044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:14:20.591698 6059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:14:20.591706 6061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:14:20.591717 6053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:14:20.592134 6044 server_base.cc:1034] running on GCE node
I20260806 21:14:20.592782 6044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:14:20.594030 6044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:14:20.595191 6044 hybrid_clock.cc:648] HybridClock initialized: now 1786050860595164 us; error 42 us; skew 500 ppm
Aug 06 21:14:20 dist-test-slave-lgr5 krb5kdc[6026](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050860, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:14:20.597945 6044 init.cc:377] Logged in from keytab as kudu/127.16.178.62@KRBTEST.COM (short username kudu)
I20260806 21:14:20.598902 6044 webserver.cc:460] Webserver started at http://127.16.178.62:45853/ using document root <none> and password file <none>
I20260806 21:14:20.599465 6044 fs_manager.cc:359] Metadata directory not provided
I20260806 21:14:20.599552 6044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:14:20.599754 6044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:14:20.601890 6044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "79032a5b3a544c7da055e5bc5021fad6"
format_stamp: "Formatted at 2026-08-06 21:14:20 on dist-test-slave-lgr5"
server_key: "42616d59e2d87e8b19a1aa355a8396a3"
server_key_iv: "2e3846b2e02292e77950972e6f34b05f"
server_key_version: "encryptionkey@0"
I20260806 21:14:20.602450 6044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "79032a5b3a544c7da055e5bc5021fad6"
format_stamp: "Formatted at 2026-08-06 21:14:20 on dist-test-slave-lgr5"
server_key: "42616d59e2d87e8b19a1aa355a8396a3"
server_key_iv: "2e3846b2e02292e77950972e6f34b05f"
server_key_version: "encryptionkey@0"
I20260806 21:14:20.606436 6044 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:14:20.608951 6068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:20.610203 6044 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 21:14:20.610330 6044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "79032a5b3a544c7da055e5bc5021fad6"
format_stamp: "Formatted at 2026-08-06 21:14:20 on dist-test-slave-lgr5"
server_key: "42616d59e2d87e8b19a1aa355a8396a3"
server_key_iv: "2e3846b2e02292e77950972e6f34b05f"
server_key_version: "encryptionkey@0"
I20260806 21:14:20.610440 6044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:14:20.626370 6044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:14:20.629638 6044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:14:20.629828 6044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:14:20.637105 6044 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.62:38989
I20260806 21:14:20.637125 6145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.62:38989 every 8 connection(s)
I20260806 21:14:20.638082 6044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:14:20.640511 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 6044
I20260806 21:14:20.640638 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:14:20.640965 17096 external_mini_cluster.cc:1428] Setting key 684b4773c8f254a1338b801f70a9bc89
I20260806 21:14:20.641144 6147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:14:20.648849 6147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79032a5b3a544c7da055e5bc5021fad6: Bootstrap starting.
Aug 06 21:14:20 dist-test-slave-lgr5 krb5kdc[6026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:14:20.651796 6147 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79032a5b3a544c7da055e5bc5021fad6: Neither blocks nor log segments found. Creating new log.
I20260806 21:14:20.652598 6147 log.cc:824] T 00000000000000000000000000000000 P 79032a5b3a544c7da055e5bc5021fad6: Log is configured to *not* fsync() on all Append() calls
I20260806 21:14:20.654922 6147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79032a5b3a544c7da055e5bc5021fad6: No bootstrap required, opened a new log
I20260806 21:14:20.656443 6154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:20.642439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46094 (local address 127.16.178.62:38989)
0806 21:14:20.642964 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:20.642981 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:20.643023 (+ 42us) server_negotiation.cc:407] Connection header received
0806 21:14:20.644155 (+ 1132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:20.644180 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:20.644589 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:20.644924 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:14:20.645945 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:20.646990 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:20.648227 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:20.648733 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:20.648855 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:20.651686 (+ 2831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:14:20.651692 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:14:20.651706 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:14:20.651740 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:14:20.653581 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:20.654062 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:20.654066 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:20.654070 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:20.654179 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:20.654443 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:20.654445 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:20.654446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:20.654857 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:14:20.655035 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:20.655345 (+ 310us) server_negotiation.cc:299] Negotiation successful
0806 21:14:20.655640 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":158,"threads_started":1}
I20260806 21:14:20.659149 6147 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79032a5b3a544c7da055e5bc5021fad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79032a5b3a544c7da055e5bc5021fad6" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 38989 } }
I20260806 21:14:20.659468 17096 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:14:20.659466 6147 raft_consensus.cc:374] T 00000000000000000000000000000000 P 79032a5b3a544c7da055e5bc5021fad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:14:20.659528 6147 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79032a5b3a544c7da055e5bc5021fad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79032a5b3a544c7da055e5bc5021fad6, State: Initialized, Role: FOLLOWER
I20260806 21:14:20.660005 6147 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79032a5b3a544c7da055e5bc5021fad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79032a5b3a544c7da055e5bc5021fad6" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 38989 } }
I20260806 21:14:20.660198 6147 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79032a5b3a544c7da055e5bc5021fad6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:14:20.660269 6147 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79032a5b3a544c7da055e5bc5021fad6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:14:20.660370 6147 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79032a5b3a544c7da055e5bc5021fad6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:14:20.661607 6147 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79032a5b3a544c7da055e5bc5021fad6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79032a5b3a544c7da055e5bc5021fad6" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 38989 } }
I20260806 21:14:20.661973 6147 leader_election.cc:302] T 00000000000000000000000000000000 P 79032a5b3a544c7da055e5bc5021fad6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79032a5b3a544c7da055e5bc5021fad6; no voters:
I20260806 21:14:20.662324 6147 leader_election.cc:288] T 00000000000000000000000000000000 P 79032a5b3a544c7da055e5bc5021fad6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:14:20.662475 6159 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79032a5b3a544c7da055e5bc5021fad6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:14:20.662700 6159 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79032a5b3a544c7da055e5bc5021fad6 [term 1 LEADER]: Becoming Leader. State: Replica: 79032a5b3a544c7da055e5bc5021fad6, State: Running, Role: LEADER
I20260806 21:14:20.663049 6159 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79032a5b3a544c7da055e5bc5021fad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79032a5b3a544c7da055e5bc5021fad6" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 38989 } }
I20260806 21:14:20.663650 6147 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79032a5b3a544c7da055e5bc5021fad6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:14:20.664951 6162 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79032a5b3a544c7da055e5bc5021fad6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79032a5b3a544c7da055e5bc5021fad6. Latest consensus state: current_term: 1 leader_uuid: "79032a5b3a544c7da055e5bc5021fad6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79032a5b3a544c7da055e5bc5021fad6" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 38989 } } }
I20260806 21:14:20.665154 6162 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79032a5b3a544c7da055e5bc5021fad6 [sys.catalog]: This master's current role is: LEADER
I20260806 21:14:20.664952 6160 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79032a5b3a544c7da055e5bc5021fad6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "79032a5b3a544c7da055e5bc5021fad6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79032a5b3a544c7da055e5bc5021fad6" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 38989 } } }
I20260806 21:14:20.665582 6160 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79032a5b3a544c7da055e5bc5021fad6 [sys.catalog]: This master's current role is: LEADER
I20260806 21:14:20.665660 6173 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:14:20.669060 6173 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:14:20.676721 6173 catalog_manager.cc:1347] Generated new cluster ID: d01776c07dc64b35881302447082b8c7
I20260806 21:14:20.676828 6173 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:14:20.684399 6173 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:14:20.685458 6173 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:14:20.693813 6173 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 79032a5b3a544c7da055e5bc5021fad6: Generated new TSK 0
I20260806 21:14:20.694710 6173 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:14:20.716593 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 6044
2026-08-06T21:14:20Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1557 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:14:20 dist-test-slave-lgr5 krb5kdc[6191](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:14:20 dist-test-slave-lgr5 krb5kdc[6191](info): set up 2 sockets
Aug 06 21:14:20 dist-test-slave-lgr5 krb5kdc[6191](info): commencing operation
krb5kdc: starting...
W20260806 21:14:21.874776 17096 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Aug 06 21:14:21 dist-test-slave-lgr5 krb5kdc[6191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:14:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:14:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.178.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
I20260806 21:14:22.016564 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:35423
--webserver_interface=127.16.178.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:46639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:35423
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:14:22.131695 6213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:14:22.131999 6213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:14:22.132056 6213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:14:22.136385 6213 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:14:22.136478 6213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:14:22.136504 6213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:14:22.136529 6213 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:14:22.136559 6213 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:14:22.142398 6213 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:46639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:35423
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:35423
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.178.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6213
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:14:22.143659 6213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:14:22.144788 6213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:14:22.153550 6220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:14:22.153550 6222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:14:22.153576 6225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:14:22.153762 6213 server_base.cc:1034] running on GCE node
I20260806 21:14:22.154541 6213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:14:22.155800 6213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:14:22.156985 6213 hybrid_clock.cc:648] HybridClock initialized: now 1786050862156955 us; error 52 us; skew 500 ppm
Aug 06 21:14:22 dist-test-slave-lgr5 krb5kdc[6191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050862, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:14:22.164721 6213 init.cc:377] Logged in from keytab as kudu/127.16.178.62@KRBTEST.COM (short username kudu)
I20260806 21:14:22.165987 6213 webserver.cc:460] Webserver started at http://127.16.178.62:42461/ using document root <none> and password file <none>
I20260806 21:14:22.166656 6213 fs_manager.cc:359] Metadata directory not provided
I20260806 21:14:22.166742 6213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:14:22.167001 6213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:14:22.169634 6213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "b5b9bac96afb4727a08a2d7f92db68ea"
format_stamp: "Formatted at 2026-08-06 21:14:22 on dist-test-slave-lgr5"
server_key: "0046ffbcfdd50ddfd0180b4c19f74b1b"
server_key_iv: "c9d1f2ed017a450a5c39e8ea9905dbff"
server_key_version: "encryptionkey@0"
I20260806 21:14:22.170472 6213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "b5b9bac96afb4727a08a2d7f92db68ea"
format_stamp: "Formatted at 2026-08-06 21:14:22 on dist-test-slave-lgr5"
server_key: "0046ffbcfdd50ddfd0180b4c19f74b1b"
server_key_iv: "c9d1f2ed017a450a5c39e8ea9905dbff"
server_key_version: "encryptionkey@0"
I20260806 21:14:22.175326 6213 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 21:14:22.178707 6237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:22.180511 6213 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:14:22.180706 6213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "b5b9bac96afb4727a08a2d7f92db68ea"
format_stamp: "Formatted at 2026-08-06 21:14:22 on dist-test-slave-lgr5"
server_key: "0046ffbcfdd50ddfd0180b4c19f74b1b"
server_key_iv: "c9d1f2ed017a450a5c39e8ea9905dbff"
server_key_version: "encryptionkey@0"
I20260806 21:14:22.180858 6213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:14:22.211651 6213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:14:22.215281 6213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:14:22.215507 6213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:14:22.222880 6213 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.62:35423
I20260806 21:14:22.223150 6327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.62:35423 every 8 connection(s)
I20260806 21:14:22.223953 6213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:14:22.225100 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 6213
I20260806 21:14:22.225224 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:14:22.225499 17096 external_mini_cluster.cc:1428] Setting key 2a6cd596d7ff27f5fa32216633dd6131
I20260806 21:14:22.227206 6330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:14:22.233987 6330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea: Bootstrap starting.
Aug 06 21:14:22 dist-test-slave-lgr5 krb5kdc[6191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:14:22.236516 6330 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea: Neither blocks nor log segments found. Creating new log.
I20260806 21:14:22.237280 6330 log.cc:824] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea: Log is configured to *not* fsync() on all Append() calls
I20260806 21:14:22.239998 6330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea: No bootstrap required, opened a new log
I20260806 21:14:22.240847 6335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:22.226838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48476 (local address 127.16.178.62:35423)
0806 21:14:22.227275 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:22.227297 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:22.227342 (+ 45us) server_negotiation.cc:407] Connection header received
0806 21:14:22.228040 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:22.228058 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:22.228496 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:22.228787 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:14:22.229752 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:22.230781 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:22.231926 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:22.232461 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:22.232582 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:22.235336 (+ 2754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:14:22.235342 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:14:22.235356 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:14:22.235407 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:14:22.237148 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:22.237792 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:22.237800 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:22.237807 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:22.237970 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:22.238342 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:22.238345 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:22.238347 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:22.238941 (+ 594us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:14:22.239142 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:22.239457 (+ 315us) server_negotiation.cc:299] Negotiation successful
0806 21:14:22.239805 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":117,"threads_started":1}
I20260806 21:14:22.244349 6330 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5b9bac96afb4727a08a2d7f92db68ea" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 35423 } }
I20260806 21:14:22.244618 6330 raft_consensus.cc:374] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:14:22.244678 6330 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5b9bac96afb4727a08a2d7f92db68ea, State: Initialized, Role: FOLLOWER
I20260806 21:14:22.245142 6330 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5b9bac96afb4727a08a2d7f92db68ea" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 35423 } }
I20260806 21:14:22.245313 6330 raft_consensus.cc:388] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:14:22.245389 6330 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:14:22.245504 6330 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:14:22.246567 6330 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5b9bac96afb4727a08a2d7f92db68ea" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 35423 } }
I20260806 21:14:22.246932 6330 leader_election.cc:302] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b5b9bac96afb4727a08a2d7f92db68ea; no voters:
I20260806 21:14:22.247278 6330 leader_election.cc:288] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:14:22.247418 6338 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:14:22.247694 6338 raft_consensus.cc:686] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [term 1 LEADER]: Becoming Leader. State: Replica: b5b9bac96afb4727a08a2d7f92db68ea, State: Running, Role: LEADER
I20260806 21:14:22.248003 6338 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5b9bac96afb4727a08a2d7f92db68ea" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 35423 } }
I20260806 21:14:22.248658 6330 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:14:22.249796 6339 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b5b9bac96afb4727a08a2d7f92db68ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5b9bac96afb4727a08a2d7f92db68ea" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 35423 } } }
I20260806 21:14:22.249943 6339 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [sys.catalog]: This master's current role is: LEADER
I20260806 21:14:22.250267 6340 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5b9bac96afb4727a08a2d7f92db68ea. Latest consensus state: current_term: 1 leader_uuid: "b5b9bac96afb4727a08a2d7f92db68ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5b9bac96afb4727a08a2d7f92db68ea" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 35423 } } }
I20260806 21:14:22.250339 6348 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:14:22.250411 6340 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [sys.catalog]: This master's current role is: LEADER
I20260806 21:14:22.254050 6348 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:14:22.260608 6348 catalog_manager.cc:1347] Generated new cluster ID: e759654df3f34f68be7a171971ffed5f
I20260806 21:14:22.260706 6348 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:14:22.279639 6348 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:14:22.280773 6348 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:14:22.288372 6348 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea: Generated new TSK 0
I20260806 21:14:22.289171 6348 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.178.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab.
Entry for principal kudu/127.16.178.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab.
I20260806 21:14:22.324961 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.1:0
--local_ip_for_outbound_sockets=127.16.178.1
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:35423
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab
--principal=kudu/127.16.178.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:46639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:14:22.446383 6366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:14:22.446709 6366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:14:22.446760 6366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:14:22.451010 6366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:14:22.451099 6366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:14:22.451172 6366 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.1
I20260806 21:14:22.456768 6366 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:46639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab
--principal=kudu/127.16.178.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:35423
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6366
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.1
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:14:22.457902 6366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:14:22.458993 6366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:14:22.470094 6374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:14:22.470119 6377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:14:22.470146 6375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:14:22.470247 6366 server_base.cc:1034] running on GCE node
I20260806 21:14:22.470903 6366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:14:22.471637 6366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:14:22.472806 6366 hybrid_clock.cc:648] HybridClock initialized: now 1786050862472782 us; error 43 us; skew 500 ppm
Aug 06 21:14:22 dist-test-slave-lgr5 krb5kdc[6191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050862, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:14:22.475926 6366 init.cc:377] Logged in from keytab as kudu/127.16.178.1@KRBTEST.COM (short username kudu)
I20260806 21:14:22.476997 6366 webserver.cc:460] Webserver started at http://127.16.178.1:35361/ using document root <none> and password file <none>
I20260806 21:14:22.477643 6366 fs_manager.cc:359] Metadata directory not provided
I20260806 21:14:22.477715 6366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:14:22.477969 6366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:14:22.480528 6366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-0/data/instance:
uuid: "5269f8bc3f5445f9af84a55eca845099"
format_stamp: "Formatted at 2026-08-06 21:14:22 on dist-test-slave-lgr5"
server_key: "7cd9a778070dd6249d86bd1408a5b3ba"
server_key_iv: "0066c2214e265ece377d8d83b9a17891"
server_key_version: "encryptionkey@0"
I20260806 21:14:22.481196 6366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance:
uuid: "5269f8bc3f5445f9af84a55eca845099"
format_stamp: "Formatted at 2026-08-06 21:14:22 on dist-test-slave-lgr5"
server_key: "7cd9a778070dd6249d86bd1408a5b3ba"
server_key_iv: "0066c2214e265ece377d8d83b9a17891"
server_key_version: "encryptionkey@0"
I20260806 21:14:22.485512 6366 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260806 21:14:22.488333 6390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:22.489806 6366 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 21:14:22.489954 6366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-0/wal
uuid: "5269f8bc3f5445f9af84a55eca845099"
format_stamp: "Formatted at 2026-08-06 21:14:22 on dist-test-slave-lgr5"
server_key: "7cd9a778070dd6249d86bd1408a5b3ba"
server_key_iv: "0066c2214e265ece377d8d83b9a17891"
server_key_version: "encryptionkey@0"
I20260806 21:14:22.490083 6366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:14:22.505106 6366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:14:22.508598 6366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:14:22.508827 6366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:14:22.509497 6366 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:14:22.510501 6366 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:14:22.510555 6366 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:22.510614 6366 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:14:22.510640 6366 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:22.520442 6366 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.1:36213
I20260806 21:14:22.520467 6567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.1:36213 every 8 connection(s)
I20260806 21:14:22.521591 6366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
I20260806 21:14:22.522909 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 6366
I20260806 21:14:22.523061 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance
I20260806 21:14:22.523360 17096 external_mini_cluster.cc:1428] Setting key 56f38d522d27fc0eb7ac973e228f9990
Aug 06 21:14:22 dist-test-slave-lgr5 krb5kdc[6191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050862, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.178.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.2@KRBTEST.COM" created.
I20260806 21:14:22.540823 6335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:22.523856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:42713 (local address 127.16.178.62:35423)
0806 21:14:22.524028 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:22.524032 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:22.525252 (+ 1220us) server_negotiation.cc:407] Connection header received
0806 21:14:22.527102 (+ 1850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:22.527105 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:22.527156 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:22.527236 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:14:22.528705 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:22.529274 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:22.530544 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:22.530781 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:22.530879 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:22.533938 (+ 3059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:14:22.533942 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:14:22.533945 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:14:22.533977 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:14:22.535579 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:22.536357 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:22.536361 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:22.536362 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:22.536467 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:22.536912 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:22.536916 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:22.536917 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:22.537213 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:14:22.537329 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:22.540424 (+ 3095us) server_negotiation.cc:299] Negotiation successful
0806 21:14:22.540634 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 21:14:22.541778 6570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:22.524385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.1:42713)
0806 21:14:22.525038 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:22.525108 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0806 21:14:22.526838 (+ 1730us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:14:22.527460 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:14:22.527470 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:14:22.527885 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:14:22.528536 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:22.528552 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:22.529412 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:22.529415 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:22.530412 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:22.530427 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:22.530961 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:22.530966 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:22.531143 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:22.531870 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:14:22.531891 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:14:22.533689 (+ 1798us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:14:22.535762 (+ 2073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:22.535771 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:22.535791 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:22.536221 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:22.536568 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:22.536571 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:22.536573 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:22.536757 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:22.539540 (+ 2783us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:14:22.539550 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:14:22.539956 (+ 406us) client_negotiation.cc:769] Sending connection context
0806 21:14:22.540252 (+ 296us) client_negotiation.cc:240] Negotiation successful
0806 21:14:22.540639 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":164,"threads_started":1}
I20260806 21:14:22.543282 6568 heartbeater.cc:347] Connected to a master server at 127.16.178.62:35423
I20260806 21:14:22.543578 6568 heartbeater.cc:464] Registering TS with master...
I20260806 21:14:22.544190 6568 heartbeater.cc:511] Master 127.16.178.62:35423 requested a full tablet report, sending...
I20260806 21:14:22.545825 6267 ts_manager.cc:194] Registered new tserver with Master: 5269f8bc3f5445f9af84a55eca845099 (127.16.178.1:36213)
I20260806 21:14:22.547025 6267 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.1@KRBTEST.COM'} at 127.16.178.1:42713
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab.
Entry for principal kudu/127.16.178.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab.
I20260806 21:14:22.556058 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.2:0
--local_ip_for_outbound_sockets=127.16.178.2
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:35423
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab
--principal=kudu/127.16.178.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:46639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:14:22.673568 6575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:14:22.673832 6575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:14:22.673875 6575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:14:22.677836 6575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:14:22.677894 6575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:14:22.677953 6575 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.2
I20260806 21:14:22.683431 6575 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:46639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab
--principal=kudu/127.16.178.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:35423
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6575
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.2
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:14:22.684623 6575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:14:22.685673 6575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:14:22.695706 6592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:14:22.695735 6586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:14:22.695735 6587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:14:22.696215 6575 server_base.cc:1034] running on GCE node
I20260806 21:14:22.696645 6575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:14:22.697322 6575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:14:22.698482 6575 hybrid_clock.cc:648] HybridClock initialized: now 1786050862698456 us; error 41 us; skew 500 ppm
Aug 06 21:14:22 dist-test-slave-lgr5 krb5kdc[6191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050862, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:14:22.701164 6575 init.cc:377] Logged in from keytab as kudu/127.16.178.2@KRBTEST.COM (short username kudu)
I20260806 21:14:22.702122 6575 webserver.cc:460] Webserver started at http://127.16.178.2:42233/ using document root <none> and password file <none>
I20260806 21:14:22.702678 6575 fs_manager.cc:359] Metadata directory not provided
I20260806 21:14:22.702744 6575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:14:22.702947 6575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:14:22.705111 6575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-1/data/instance:
uuid: "e2779b8373194b8ba7a735b2c5c20c15"
format_stamp: "Formatted at 2026-08-06 21:14:22 on dist-test-slave-lgr5"
server_key: "615f7c32a01ce888513a082d28ad21ff"
server_key_iv: "3804dc8dbd34e394e35749f1a656045e"
server_key_version: "encryptionkey@0"
I20260806 21:14:22.705647 6575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance:
uuid: "e2779b8373194b8ba7a735b2c5c20c15"
format_stamp: "Formatted at 2026-08-06 21:14:22 on dist-test-slave-lgr5"
server_key: "615f7c32a01ce888513a082d28ad21ff"
server_key_iv: "3804dc8dbd34e394e35749f1a656045e"
server_key_version: "encryptionkey@0"
I20260806 21:14:22.709431 6575 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:14:22.713877 6609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:22.715301 6575 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:14:22.715461 6575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-1/wal
uuid: "e2779b8373194b8ba7a735b2c5c20c15"
format_stamp: "Formatted at 2026-08-06 21:14:22 on dist-test-slave-lgr5"
server_key: "615f7c32a01ce888513a082d28ad21ff"
server_key_iv: "3804dc8dbd34e394e35749f1a656045e"
server_key_version: "encryptionkey@0"
I20260806 21:14:22.715581 6575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:14:22.732286 6575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:14:22.735563 6575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:14:22.735786 6575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:14:22.736478 6575 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:14:22.737413 6575 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:14:22.737465 6575 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:22.737522 6575 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:14:22.737551 6575 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:22.746464 6575 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.2:38433
I20260806 21:14:22.746497 6784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.2:38433 every 8 connection(s)
I20260806 21:14:22.747394 6575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
I20260806 21:14:22.754352 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 6575
I20260806 21:14:22.754493 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance
I20260806 21:14:22.754773 17096 external_mini_cluster.cc:1428] Setting key 4b7556188a36c2a27b10220702870bd5
Aug 06 21:14:22 dist-test-slave-lgr5 krb5kdc[6191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050862, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.2@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:14:22.761711 6335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:22.749279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:34803 (local address 127.16.178.62:35423)
0806 21:14:22.749439 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:22.749445 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:22.750369 (+ 924us) server_negotiation.cc:407] Connection header received
0806 21:14:22.751865 (+ 1496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:22.751868 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:22.751918 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:22.751981 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:14:22.753184 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:22.753706 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:22.755149 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:22.755448 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:22.755545 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:22.758191 (+ 2646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:14:22.758194 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:14:22.758197 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:14:22.758225 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:14:22.759570 (+ 1345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:22.760149 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:22.760152 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:22.760153 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:22.760234 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:22.760571 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:22.760574 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:22.760576 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:22.760766 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:14:22.760854 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:22.761409 (+ 555us) server_negotiation.cc:299] Negotiation successful
0806 21:14:22.761542 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 21:14:22.762598 6788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:22.749699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.2:34803)
0806 21:14:22.750184 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:22.750227 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 21:14:22.751639 (+ 1412us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:14:22.752150 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:14:22.752157 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:14:22.752532 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:14:22.753055 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:22.753069 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:22.753802 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:22.753804 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:22.755015 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:22.755042 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:22.755605 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:22.755611 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:22.755779 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:22.756375 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:14:22.756392 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:14:22.758008 (+ 1616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:14:22.759694 (+ 1686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:22.759701 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:22.759720 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:22.760006 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:22.760321 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:22.760324 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:22.760326 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:22.760477 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:22.760876 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:14:22.760884 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:14:22.761157 (+ 273us) client_negotiation.cc:769] Sending connection context
0806 21:14:22.761339 (+ 182us) client_negotiation.cc:240] Negotiation successful
0806 21:14:22.761695 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":108,"threads_started":1}
I20260806 21:14:22.764057 6786 heartbeater.cc:347] Connected to a master server at 127.16.178.62:35423
I20260806 21:14:22.764396 6786 heartbeater.cc:464] Registering TS with master...
I20260806 21:14:22.765069 6786 heartbeater.cc:511] Master 127.16.178.62:35423 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.16.178.3@KRBTEST.COM; defaulting to no policy
I20260806 21:14:22.766505 6267 ts_manager.cc:194] Registered new tserver with Master: e2779b8373194b8ba7a735b2c5c20c15 (127.16.178.2:38433)
I20260806 21:14:22.767144 6267 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.2@KRBTEST.COM'} at 127.16.178.2:34803
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab.
Entry for principal kudu/127.16.178.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab.
I20260806 21:14:22.787655 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.3:0
--local_ip_for_outbound_sockets=127.16.178.3
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:35423
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab
--principal=kudu/127.16.178.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:46639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:14:22.903425 6793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:14:22.903741 6793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:14:22.903786 6793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:14:22.908012 6793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:14:22.908136 6793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:14:22.908224 6793 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.3
I20260806 21:14:22.913761 6793 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:46639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab
--principal=kudu/127.16.178.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:35423
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6793
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.3
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:14:22.914880 6793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:14:22.915951 6793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 21:14:22.925828 6793 server_base.cc:1034] running on GCE node
W20260806 21:14:22.925786 6804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:14:22.925840 6803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:14:22.925786 6806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:14:22.926465 6793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:14:22.927162 6793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:14:22.928318 6793 hybrid_clock.cc:648] HybridClock initialized: now 1786050862928300 us; error 31 us; skew 500 ppm
Aug 06 21:14:22 dist-test-slave-lgr5 krb5kdc[6191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050862, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:14:22.931141 6793 init.cc:377] Logged in from keytab as kudu/127.16.178.3@KRBTEST.COM (short username kudu)
I20260806 21:14:22.932070 6793 webserver.cc:460] Webserver started at http://127.16.178.3:45195/ using document root <none> and password file <none>
I20260806 21:14:22.932674 6793 fs_manager.cc:359] Metadata directory not provided
I20260806 21:14:22.932735 6793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:14:22.932910 6793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:14:22.935067 6793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-2/data/instance:
uuid: "07b6bdd51f164475a3477bad2ca26fed"
format_stamp: "Formatted at 2026-08-06 21:14:22 on dist-test-slave-lgr5"
server_key: "a09fc69c01e6c42d66c3a02942325f84"
server_key_iv: "5da7c4883bf97482abb5b8fb88741fa9"
server_key_version: "encryptionkey@0"
I20260806 21:14:22.935624 6793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance:
uuid: "07b6bdd51f164475a3477bad2ca26fed"
format_stamp: "Formatted at 2026-08-06 21:14:22 on dist-test-slave-lgr5"
server_key: "a09fc69c01e6c42d66c3a02942325f84"
server_key_iv: "5da7c4883bf97482abb5b8fb88741fa9"
server_key_version: "encryptionkey@0"
I20260806 21:14:22.941706 6793 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260806 21:14:22.944964 6813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:22.946290 6793 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 21:14:22.946426 6793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-2/wal
uuid: "07b6bdd51f164475a3477bad2ca26fed"
format_stamp: "Formatted at 2026-08-06 21:14:22 on dist-test-slave-lgr5"
server_key: "a09fc69c01e6c42d66c3a02942325f84"
server_key_iv: "5da7c4883bf97482abb5b8fb88741fa9"
server_key_version: "encryptionkey@0"
I20260806 21:14:22.946556 6793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:14:22.959192 6793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:14:22.962462 6793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:14:22.962653 6793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:14:22.963307 6793 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:14:22.964265 6793 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:14:22.964322 6793 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:22.964380 6793 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:14:22.964406 6793 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:22.973745 6793 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.3:33729
I20260806 21:14:22.973996 6991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.3:33729 every 8 connection(s)
I20260806 21:14:22.975085 6793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
I20260806 21:14:22.975291 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 6793
I20260806 21:14:22.975412 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance
I20260806 21:14:22.975724 17096 external_mini_cluster.cc:1428] Setting key 8ab5ecb62bccee074ce98a03681875ae
Aug 06 21:14:22 dist-test-slave-lgr5 krb5kdc[6191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050862, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.3@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:14:22.996004 6335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:22.979428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:42925 (local address 127.16.178.62:35423)
0806 21:14:22.979601 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:22.979607 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:22.979625 (+ 18us) server_negotiation.cc:407] Connection header received
0806 21:14:22.980972 (+ 1347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:22.980976 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:22.981053 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:22.981131 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:14:22.982918 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:22.983646 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:22.985555 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:22.985873 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:22.985979 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:22.989246 (+ 3267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:14:22.989250 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:14:22.989253 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:14:22.989288 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:14:22.993209 (+ 3921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:22.993898 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:22.993902 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:22.993903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:22.994005 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:22.994514 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:22.994518 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:22.994520 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:22.994774 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:14:22.994888 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:22.995659 (+ 771us) server_negotiation.cc:299] Negotiation successful
0806 21:14:22.995823 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 21:14:22.997102 6997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:22.978232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.3:42925)
0806 21:14:22.978816 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:22.978878 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 21:14:22.980633 (+ 1755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:14:22.981368 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:14:22.981380 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:14:22.981955 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:14:22.982702 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:22.982728 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:22.983859 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:22.983864 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:22.985412 (+ 1548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:22.985426 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:22.986057 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:22.986061 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:22.986273 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:22.986972 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:14:22.986995 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:14:22.989011 (+ 2016us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:14:22.993374 (+ 4363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:22.993382 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:22.993400 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:22.993735 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:22.994111 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:22.994115 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:22.994117 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:22.994355 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:22.994913 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:14:22.994921 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:14:22.995316 (+ 395us) client_negotiation.cc:769] Sending connection context
0806 21:14:22.995561 (+ 245us) client_negotiation.cc:240] Negotiation successful
0806 21:14:22.995975 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":131,"threads_started":1}
I20260806 21:14:22.998373 6993 heartbeater.cc:347] Connected to a master server at 127.16.178.62:35423
I20260806 21:14:22.998693 6993 heartbeater.cc:464] Registering TS with master...
I20260806 21:14:22.999425 6993 heartbeater.cc:511] Master 127.16.178.62:35423 requested a full tablet report, sending...
I20260806 21:14:23.002060 6265 ts_manager.cc:194] Registered new tserver with Master: 07b6bdd51f164475a3477bad2ca26fed (127.16.178.3:33729)
I20260806 21:14:23.002812 6265 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.3@KRBTEST.COM'} at 127.16.178.3:42925
I20260806 21:14:23.003000 17096 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 21:14:23.013379 6335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:23.005484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48482 (local address 127.16.178.62:35423)
0806 21:14:23.005626 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:23.005630 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:23.005752 (+ 122us) server_negotiation.cc:407] Connection header received
0806 21:14:23.005923 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:23.005927 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:23.005984 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:23.006085 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:14:23.006875 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:23.007415 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:23.008436 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:23.008670 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:23.008767 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:23.010153 (+ 1386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:14:23.010156 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:14:23.010159 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:14:23.010192 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:14:23.011559 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:23.012212 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:23.012215 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:23.012216 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:23.012308 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:23.012568 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:23.012570 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:23.012572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:23.012808 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:14:23.012935 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:23.013135 (+ 200us) server_negotiation.cc:299] Negotiation successful
0806 21:14:23.013247 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 21:14:23.016729 6265 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48482:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:14:23.019919 6265 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:14:23.034374 7011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:23.028909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35804 (local address 127.16.178.1:36213)
0806 21:14:23.029844 (+ 935us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:23.029860 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:23.029886 (+ 26us) server_negotiation.cc:407] Connection header received
0806 21:14:23.029970 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:23.029976 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:23.030190 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:23.030715 (+ 525us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:14:23.030866 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:23.031713 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:23.033083 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:23.033676 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:23.033781 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:23.033957 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:23.034104 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 21:14:23.034202 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":826,"thread_start_us":112,"threads_started":1}
I20260806 21:14:23.034363 7009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:23.028724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:36213 (local address 127.0.0.1:35804)
0806 21:14:23.029203 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:23.029252 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 21:14:23.029399 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:14:23.030381 (+ 982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:14:23.030387 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:14:23.030414 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:14:23.030576 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:23.030584 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:23.031845 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:23.031848 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:23.032923 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:23.032935 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:23.033797 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:23.033800 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:23.033915 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:23.033954 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 21:14:23.034065 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 21:14:23.034199 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":115,"threads_started":1}
I20260806 21:14:23.035172 7013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:23.028905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.2:38433 (local address 127.0.0.1:53356)
0806 21:14:23.029716 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:23.029731 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:14:23.029853 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:14:23.030677 (+ 824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:14:23.030680 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:14:23.030696 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:14:23.030831 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:23.030841 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:23.032421 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:23.032424 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:23.033448 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:23.033459 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:23.034786 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:23.034790 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:23.034882 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:23.034906 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 21:14:23.034965 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 21:14:23.035046 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":742,"mutex_wait_us":41,"thread_start_us":110,"threads_started":1}
I20260806 21:14:23.035418 7015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:23.029564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53356 (local address 127.16.178.2:38433)
0806 21:14:23.030273 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:23.030285 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:23.030307 (+ 22us) server_negotiation.cc:407] Connection header received
0806 21:14:23.030377 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:23.030381 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:23.030545 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:23.030703 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:14:23.030978 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:23.032244 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:23.033580 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:23.034616 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:23.034774 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:23.034929 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:23.035014 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 21:14:23.035131 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":612,"thread_start_us":120,"threads_started":1}
I20260806 21:14:23.036445 7014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:23.029456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.3:33729 (local address 127.0.0.1:56828)
0806 21:14:23.030323 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:23.030338 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:14:23.030437 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:14:23.030906 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:14:23.030909 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:14:23.030927 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:14:23.031003 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:23.031009 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:23.032421 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:23.032424 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:23.034603 (+ 2179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:23.034614 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:23.035973 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:23.035978 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:23.036071 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:23.036129 (+ 58us) client_negotiation.cc:769] Sending connection context
0806 21:14:23.036218 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 21:14:23.036296 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":793,"mutex_wait_us":41,"thread_start_us":92,"threads_started":1}
I20260806 21:14:23.036984 7016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:23.029753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56828 (local address 127.16.178.3:33729)
0806 21:14:23.030365 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:23.030380 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:23.030408 (+ 28us) server_negotiation.cc:407] Connection header received
0806 21:14:23.030554 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:23.030560 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:23.030731 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:23.030902 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:14:23.031101 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:23.032250 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:23.034767 (+ 2517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:23.035624 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:23.035741 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:23.035899 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:23.036700 (+ 801us) server_negotiation.cc:299] Negotiation successful
0806 21:14:23.036823 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":204,"threads_started":1}
I20260806 21:14:23.037925 6455 tablet_service.cc:1467] Processing CreateTablet for tablet b921a9a719184abcba8006aaf14735d8 (DEFAULT_TABLE table=test-table [id=be7e984ca62247d6a85861aad523994e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:14:23.038146 6688 tablet_service.cc:1467] Processing CreateTablet for tablet b921a9a719184abcba8006aaf14735d8 (DEFAULT_TABLE table=test-table [id=be7e984ca62247d6a85861aad523994e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:14:23.039093 6688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b921a9a719184abcba8006aaf14735d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:14:23.039111 6455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b921a9a719184abcba8006aaf14735d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:14:23.039619 6878 tablet_service.cc:1467] Processing CreateTablet for tablet b921a9a719184abcba8006aaf14735d8 (DEFAULT_TABLE table=test-table [id=be7e984ca62247d6a85861aad523994e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:14:23.040879 6878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b921a9a719184abcba8006aaf14735d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:14:23.046831 7018 tablet_bootstrap.cc:492] T b921a9a719184abcba8006aaf14735d8 P e2779b8373194b8ba7a735b2c5c20c15: Bootstrap starting.
I20260806 21:14:23.047166 7020 tablet_bootstrap.cc:492] T b921a9a719184abcba8006aaf14735d8 P 5269f8bc3f5445f9af84a55eca845099: Bootstrap starting.
I20260806 21:14:23.047851 7021 tablet_bootstrap.cc:492] T b921a9a719184abcba8006aaf14735d8 P 07b6bdd51f164475a3477bad2ca26fed: Bootstrap starting.
I20260806 21:14:23.049643 7020 tablet_bootstrap.cc:654] T b921a9a719184abcba8006aaf14735d8 P 5269f8bc3f5445f9af84a55eca845099: Neither blocks nor log segments found. Creating new log.
I20260806 21:14:23.049968 7021 tablet_bootstrap.cc:654] T b921a9a719184abcba8006aaf14735d8 P 07b6bdd51f164475a3477bad2ca26fed: Neither blocks nor log segments found. Creating new log.
I20260806 21:14:23.050336 7018 tablet_bootstrap.cc:654] T b921a9a719184abcba8006aaf14735d8 P e2779b8373194b8ba7a735b2c5c20c15: Neither blocks nor log segments found. Creating new log.
I20260806 21:14:23.050720 7020 log.cc:824] T b921a9a719184abcba8006aaf14735d8 P 5269f8bc3f5445f9af84a55eca845099: Log is configured to *not* fsync() on all Append() calls
I20260806 21:14:23.051664 7021 log.cc:824] T b921a9a719184abcba8006aaf14735d8 P 07b6bdd51f164475a3477bad2ca26fed: Log is configured to *not* fsync() on all Append() calls
I20260806 21:14:23.051743 7018 log.cc:824] T b921a9a719184abcba8006aaf14735d8 P e2779b8373194b8ba7a735b2c5c20c15: Log is configured to *not* fsync() on all Append() calls
I20260806 21:14:23.053519 7020 tablet_bootstrap.cc:492] T b921a9a719184abcba8006aaf14735d8 P 5269f8bc3f5445f9af84a55eca845099: No bootstrap required, opened a new log
I20260806 21:14:23.053685 7021 tablet_bootstrap.cc:492] T b921a9a719184abcba8006aaf14735d8 P 07b6bdd51f164475a3477bad2ca26fed: No bootstrap required, opened a new log
I20260806 21:14:23.053746 7020 ts_tablet_manager.cc:1397] T b921a9a719184abcba8006aaf14735d8 P 5269f8bc3f5445f9af84a55eca845099: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 21:14:23.053896 7021 ts_tablet_manager.cc:1397] T b921a9a719184abcba8006aaf14735d8 P 07b6bdd51f164475a3477bad2ca26fed: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 21:14:23.054117 7018 tablet_bootstrap.cc:492] T b921a9a719184abcba8006aaf14735d8 P e2779b8373194b8ba7a735b2c5c20c15: No bootstrap required, opened a new log
I20260806 21:14:23.054301 7018 ts_tablet_manager.cc:1397] T b921a9a719184abcba8006aaf14735d8 P e2779b8373194b8ba7a735b2c5c20c15: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260806 21:14:23.057406 7020 raft_consensus.cc:348] T b921a9a719184abcba8006aaf14735d8 P 5269f8bc3f5445f9af84a55eca845099 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5269f8bc3f5445f9af84a55eca845099" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36213 } } peers { permanent_uuid: "e2779b8373194b8ba7a735b2c5c20c15" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38433 } } peers { permanent_uuid: "07b6bdd51f164475a3477bad2ca26fed" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33729 } }
I20260806 21:14:23.057657 7020 raft_consensus.cc:374] T b921a9a719184abcba8006aaf14735d8 P 5269f8bc3f5445f9af84a55eca845099 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:14:23.057716 7020 raft_consensus.cc:729] T b921a9a719184abcba8006aaf14735d8 P 5269f8bc3f5445f9af84a55eca845099 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5269f8bc3f5445f9af84a55eca845099, State: Initialized, Role: FOLLOWER
I20260806 21:14:23.057812 7021 raft_consensus.cc:348] T b921a9a719184abcba8006aaf14735d8 P 07b6bdd51f164475a3477bad2ca26fed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5269f8bc3f5445f9af84a55eca845099" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36213 } } peers { permanent_uuid: "e2779b8373194b8ba7a735b2c5c20c15" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38433 } } peers { permanent_uuid: "07b6bdd51f164475a3477bad2ca26fed" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33729 } }
I20260806 21:14:23.057902 7018 raft_consensus.cc:348] T b921a9a719184abcba8006aaf14735d8 P e2779b8373194b8ba7a735b2c5c20c15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5269f8bc3f5445f9af84a55eca845099" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36213 } } peers { permanent_uuid: "e2779b8373194b8ba7a735b2c5c20c15" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38433 } } peers { permanent_uuid: "07b6bdd51f164475a3477bad2ca26fed" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33729 } }
I20260806 21:14:23.058096 7018 raft_consensus.cc:374] T b921a9a719184abcba8006aaf14735d8 P e2779b8373194b8ba7a735b2c5c20c15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:14:23.058090 7021 raft_consensus.cc:374] T b921a9a719184abcba8006aaf14735d8 P 07b6bdd51f164475a3477bad2ca26fed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:14:23.058147 7018 raft_consensus.cc:729] T b921a9a719184abcba8006aaf14735d8 P e2779b8373194b8ba7a735b2c5c20c15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2779b8373194b8ba7a735b2c5c20c15, State: Initialized, Role: FOLLOWER
I20260806 21:14:23.058147 7021 raft_consensus.cc:729] T b921a9a719184abcba8006aaf14735d8 P 07b6bdd51f164475a3477bad2ca26fed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b6bdd51f164475a3477bad2ca26fed, State: Initialized, Role: FOLLOWER
I20260806 21:14:23.058197 7020 consensus_queue.cc:260] T b921a9a719184abcba8006aaf14735d8 P 5269f8bc3f5445f9af84a55eca845099 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5269f8bc3f5445f9af84a55eca845099" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36213 } } peers { permanent_uuid: "e2779b8373194b8ba7a735b2c5c20c15" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38433 } } peers { permanent_uuid: "07b6bdd51f164475a3477bad2ca26fed" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33729 } }
I20260806 21:14:23.058573 7021 consensus_queue.cc:260] T b921a9a719184abcba8006aaf14735d8 P 07b6bdd51f164475a3477bad2ca26fed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5269f8bc3f5445f9af84a55eca845099" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36213 } } peers { permanent_uuid: "e2779b8373194b8ba7a735b2c5c20c15" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38433 } } peers { permanent_uuid: "07b6bdd51f164475a3477bad2ca26fed" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33729 } }
I20260806 21:14:23.058806 7018 consensus_queue.cc:260] T b921a9a719184abcba8006aaf14735d8 P e2779b8373194b8ba7a735b2c5c20c15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5269f8bc3f5445f9af84a55eca845099" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36213 } } peers { permanent_uuid: "e2779b8373194b8ba7a735b2c5c20c15" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38433 } } peers { permanent_uuid: "07b6bdd51f164475a3477bad2ca26fed" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33729 } }
I20260806 21:14:23.059017 6568 heartbeater.cc:503] Master 127.16.178.62:35423 was elected leader, sending a full tablet report...
I20260806 21:14:23.059332 7020 ts_tablet_manager.cc:1428] T b921a9a719184abcba8006aaf14735d8 P 5269f8bc3f5445f9af84a55eca845099: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 21:14:23.059726 7018 ts_tablet_manager.cc:1428] T b921a9a719184abcba8006aaf14735d8 P e2779b8373194b8ba7a735b2c5c20c15: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 21:14:23.060043 6786 heartbeater.cc:503] Master 127.16.178.62:35423 was elected leader, sending a full tablet report...
I20260806 21:14:23.060153 6993 heartbeater.cc:503] Master 127.16.178.62:35423 was elected leader, sending a full tablet report...
I20260806 21:14:23.060417 7021 ts_tablet_manager.cc:1428] T b921a9a719184abcba8006aaf14735d8 P 07b6bdd51f164475a3477bad2ca26fed: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 21:14:23.108899 7028 raft_consensus.cc:482] T b921a9a719184abcba8006aaf14735d8 P 07b6bdd51f164475a3477bad2ca26fed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:14:23.109130 7028 raft_consensus.cc:504] T b921a9a719184abcba8006aaf14735d8 P 07b6bdd51f164475a3477bad2ca26fed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5269f8bc3f5445f9af84a55eca845099" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36213 } } peers { permanent_uuid: "e2779b8373194b8ba7a735b2c5c20c15" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38433 } } peers { permanent_uuid: "07b6bdd51f164475a3477bad2ca26fed" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33729 } }
I20260806 21:14:23.110328 7028 leader_election.cc:288] T b921a9a719184abcba8006aaf14735d8 P 07b6bdd51f164475a3477bad2ca26fed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5269f8bc3f5445f9af84a55eca845099 (127.16.178.1:36213), e2779b8373194b8ba7a735b2c5c20c15 (127.16.178.2:38433)
I20260806 21:14:23.114187 6997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:23.110569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.2:38433 (local address 127.16.178.3:55721)
0806 21:14:23.110685 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:23.110702 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 21:14:23.110799 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:14:23.111091 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:14:23.111093 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:14:23.111112 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:14:23.111246 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:23.111254 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:23.112073 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:23.112108 (+ 35us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:23.112992 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:23.113003 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:23.113763 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:23.113767 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:23.113889 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:23.113919 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 21:14:23.113986 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 21:14:23.114063 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 21:14:23.114218 7015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:23.110715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:55721 (local address 127.16.178.2:38433)
0806 21:14:23.110852 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:23.110856 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:23.110874 (+ 18us) server_negotiation.cc:407] Connection header received
0806 21:14:23.110935 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:23.110938 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:23.110989 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:23.111076 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:14:23.111365 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:23.111965 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:23.113147 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:23.113645 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:23.113744 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:23.113788 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:23.114014 (+ 226us) server_negotiation.cc:299] Negotiation successful
0806 21:14:23.114090 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 21:14:23.114415 7011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:23.110733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:41871 (local address 127.16.178.1:36213)
0806 21:14:23.110867 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:23.110872 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:23.111180 (+ 308us) server_negotiation.cc:407] Connection header received
0806 21:14:23.111278 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:23.111281 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:23.111359 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:23.111420 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:14:23.111683 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:23.112273 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:23.113398 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:23.113844 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:23.113957 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:23.114007 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:23.114210 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 21:14:23.114292 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 21:14:23.114744 7029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:23.110569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:36213 (local address 127.16.178.3:41871)
0806 21:14:23.111076 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:23.111091 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:14:23.111191 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:14:23.111449 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:14:23.111452 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:14:23.111470 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:14:23.111583 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:23.111590 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:23.112412 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:23.112415 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:23.113246 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:23.113257 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:23.113963 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:23.113967 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:23.114075 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:23.114099 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 21:14:23.114172 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 21:14:23.114233 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"mutex_wait_us":133,"thread_start_us":155,"threads_started":1}
I20260806 21:14:23.114943 6714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b921a9a719184abcba8006aaf14735d8" candidate_uuid: "07b6bdd51f164475a3477bad2ca26fed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2779b8373194b8ba7a735b2c5c20c15" is_pre_election: true
I20260806 21:14:23.115298 6714 raft_consensus.cc:2454] T b921a9a719184abcba8006aaf14735d8 P e2779b8373194b8ba7a735b2c5c20c15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07b6bdd51f164475a3477bad2ca26fed in term 0.
I20260806 21:14:23.115329 6482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b921a9a719184abcba8006aaf14735d8" candidate_uuid: "07b6bdd51f164475a3477bad2ca26fed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5269f8bc3f5445f9af84a55eca845099" is_pre_election: true
I20260806 21:14:23.115634 6482 raft_consensus.cc:2454] T b921a9a719184abcba8006aaf14735d8 P 5269f8bc3f5445f9af84a55eca845099 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07b6bdd51f164475a3477bad2ca26fed in term 0.
I20260806 21:14:23.115832 6817 leader_election.cc:302] T b921a9a719184abcba8006aaf14735d8 P 07b6bdd51f164475a3477bad2ca26fed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b6bdd51f164475a3477bad2ca26fed, e2779b8373194b8ba7a735b2c5c20c15; no voters:
I20260806 21:14:23.116146 7028 raft_consensus.cc:2781] T b921a9a719184abcba8006aaf14735d8 P 07b6bdd51f164475a3477bad2ca26fed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:14:23.116245 7028 raft_consensus.cc:482] T b921a9a719184abcba8006aaf14735d8 P 07b6bdd51f164475a3477bad2ca26fed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:14:23.116295 7028 raft_consensus.cc:3037] T b921a9a719184abcba8006aaf14735d8 P 07b6bdd51f164475a3477bad2ca26fed [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:14:23.117437 7028 raft_consensus.cc:504] T b921a9a719184abcba8006aaf14735d8 P 07b6bdd51f164475a3477bad2ca26fed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5269f8bc3f5445f9af84a55eca845099" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36213 } } peers { permanent_uuid: "e2779b8373194b8ba7a735b2c5c20c15" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38433 } } peers { permanent_uuid: "07b6bdd51f164475a3477bad2ca26fed" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33729 } }
I20260806 21:14:23.117830 7028 leader_election.cc:288] T b921a9a719184abcba8006aaf14735d8 P 07b6bdd51f164475a3477bad2ca26fed [CANDIDATE]: Term 1 election: Requested vote from peers 5269f8bc3f5445f9af84a55eca845099 (127.16.178.1:36213), e2779b8373194b8ba7a735b2c5c20c15 (127.16.178.2:38433)
I20260806 21:14:23.118387 6482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b921a9a719184abcba8006aaf14735d8" candidate_uuid: "07b6bdd51f164475a3477bad2ca26fed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5269f8bc3f5445f9af84a55eca845099"
I20260806 21:14:23.118556 6482 raft_consensus.cc:3037] T b921a9a719184abcba8006aaf14735d8 P 5269f8bc3f5445f9af84a55eca845099 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:14:23.118590 6714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b921a9a719184abcba8006aaf14735d8" candidate_uuid: "07b6bdd51f164475a3477bad2ca26fed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2779b8373194b8ba7a735b2c5c20c15"
I20260806 21:14:23.118748 6714 raft_consensus.cc:3037] T b921a9a719184abcba8006aaf14735d8 P e2779b8373194b8ba7a735b2c5c20c15 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:14:23.119885 6714 raft_consensus.cc:2454] T b921a9a719184abcba8006aaf14735d8 P e2779b8373194b8ba7a735b2c5c20c15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b6bdd51f164475a3477bad2ca26fed in term 1.
I20260806 21:14:23.119890 6482 raft_consensus.cc:2454] T b921a9a719184abcba8006aaf14735d8 P 5269f8bc3f5445f9af84a55eca845099 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07b6bdd51f164475a3477bad2ca26fed in term 1.
I20260806 21:14:23.120293 6814 leader_election.cc:302] T b921a9a719184abcba8006aaf14735d8 P 07b6bdd51f164475a3477bad2ca26fed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b6bdd51f164475a3477bad2ca26fed, 5269f8bc3f5445f9af84a55eca845099; no voters:
I20260806 21:14:23.120585 7028 raft_consensus.cc:2781] T b921a9a719184abcba8006aaf14735d8 P 07b6bdd51f164475a3477bad2ca26fed [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:14:23.120826 7028 raft_consensus.cc:686] T b921a9a719184abcba8006aaf14735d8 P 07b6bdd51f164475a3477bad2ca26fed [term 1 LEADER]: Becoming Leader. State: Replica: 07b6bdd51f164475a3477bad2ca26fed, State: Running, Role: LEADER
I20260806 21:14:23.121333 7028 consensus_queue.cc:237] T b921a9a719184abcba8006aaf14735d8 P 07b6bdd51f164475a3477bad2ca26fed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5269f8bc3f5445f9af84a55eca845099" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36213 } } peers { permanent_uuid: "e2779b8373194b8ba7a735b2c5c20c15" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38433 } } peers { permanent_uuid: "07b6bdd51f164475a3477bad2ca26fed" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33729 } }
I20260806 21:14:23.124984 6267 catalog_manager.cc:5495] T b921a9a719184abcba8006aaf14735d8 P 07b6bdd51f164475a3477bad2ca26fed reported cstate change: term changed from 0 to 1, leader changed from <none> to 07b6bdd51f164475a3477bad2ca26fed (127.16.178.3). New cstate: current_term: 1 leader_uuid: "07b6bdd51f164475a3477bad2ca26fed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5269f8bc3f5445f9af84a55eca845099" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2779b8373194b8ba7a735b2c5c20c15" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07b6bdd51f164475a3477bad2ca26fed" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33729 } health_report { overall_health: HEALTHY } } }
I20260806 21:14:23.160254 7016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:23.156685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56842 (local address 127.16.178.3:33729)
0806 21:14:23.156829 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:23.156833 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:23.156847 (+ 14us) server_negotiation.cc:407] Connection header received
0806 21:14:23.156963 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:23.156965 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:23.157022 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:23.157113 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:14:23.157336 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:23.157845 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:23.158714 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:23.158947 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:23.159029 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:23.159307 (+ 278us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:14:23.159782 (+ 475us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:14:23.159879 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:23.159973 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 21:14:23.160031 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 21:14:23.166633 6482 raft_consensus.cc:1261] T b921a9a719184abcba8006aaf14735d8 P 5269f8bc3f5445f9af84a55eca845099 [term 1 FOLLOWER]: Refusing update from remote peer 07b6bdd51f164475a3477bad2ca26fed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:14:23.166633 6714 raft_consensus.cc:1261] T b921a9a719184abcba8006aaf14735d8 P e2779b8373194b8ba7a735b2c5c20c15 [term 1 FOLLOWER]: Refusing update from remote peer 07b6bdd51f164475a3477bad2ca26fed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:14:23.167366 7028 consensus_queue.cc:1035] T b921a9a719184abcba8006aaf14735d8 P 07b6bdd51f164475a3477bad2ca26fed [LEADER]: Connected to new peer: Peer: permanent_uuid: "5269f8bc3f5445f9af84a55eca845099" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:14:23.167582 7030 consensus_queue.cc:1035] T b921a9a719184abcba8006aaf14735d8 P 07b6bdd51f164475a3477bad2ca26fed [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2779b8373194b8ba7a735b2c5c20c15" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:14:23.175596 7035 mvcc.cc:205] Tried to move back new op lower bound from 7315664335522902016 to 7315664335348244480. Current Snapshot: MvccSnapshot[applied={T|T < 7315664335522902016}]
I20260806 21:14:23.175698 7034 mvcc.cc:205] Tried to move back new op lower bound from 7315664335522902016 to 7315664335348244480. Current Snapshot: MvccSnapshot[applied={T|T < 7315664335522902016}]
I20260806 21:14:23.184118 6267 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48482:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:14:23.184338 6267 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48482:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:14:23.187222 6267 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea: Sending DeleteTablet for 3 replicas of tablet b921a9a719184abcba8006aaf14735d8
I20260806 21:14:23.188231 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 6366
I20260806 21:14:23.188232 6688 tablet_service.cc:1514] Processing DeleteTablet for tablet b921a9a719184abcba8006aaf14735d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:14:23 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:53356
I20260806 21:14:23.188455 6878 tablet_service.cc:1514] Processing DeleteTablet for tablet b921a9a719184abcba8006aaf14735d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:14:23 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:56828
I20260806 21:14:23.198068 7048 tablet_replica.cc:330] T b921a9a719184abcba8006aaf14735d8 P e2779b8373194b8ba7a735b2c5c20c15: stopping tablet replica
I20260806 21:14:23.198529 7048 raft_consensus.cc:2229] T b921a9a719184abcba8006aaf14735d8 P e2779b8373194b8ba7a735b2c5c20c15 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:14:23.198961 7048 raft_consensus.cc:2258] T b921a9a719184abcba8006aaf14735d8 P e2779b8373194b8ba7a735b2c5c20c15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:14:23.201332 7048 ts_tablet_manager.cc:1905] T b921a9a719184abcba8006aaf14735d8 P e2779b8373194b8ba7a735b2c5c20c15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:14:23.203384 7049 tablet_replica.cc:330] T b921a9a719184abcba8006aaf14735d8 P 07b6bdd51f164475a3477bad2ca26fed: stopping tablet replica
I20260806 21:14:23.203924 7049 raft_consensus.cc:2229] T b921a9a719184abcba8006aaf14735d8 P 07b6bdd51f164475a3477bad2ca26fed [term 1 LEADER]: Raft consensus shutting down.
I20260806 21:14:23.204032 7048 ts_tablet_manager.cc:1918] T b921a9a719184abcba8006aaf14735d8 P e2779b8373194b8ba7a735b2c5c20c15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:14:23.204164 7048 log.cc:1196] T b921a9a719184abcba8006aaf14735d8 P e2779b8373194b8ba7a735b2c5c20c15: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-1/wal/wals/b921a9a719184abcba8006aaf14735d8
I20260806 21:14:23.204572 7048 ts_tablet_manager.cc:1939] T b921a9a719184abcba8006aaf14735d8 P e2779b8373194b8ba7a735b2c5c20c15: Deleting consensus metadata
I20260806 21:14:23.204828 7049 raft_consensus.cc:2258] T b921a9a719184abcba8006aaf14735d8 P 07b6bdd51f164475a3477bad2ca26fed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:14:23.206300 6241 catalog_manager.cc:4841] TS e2779b8373194b8ba7a735b2c5c20c15 (127.16.178.2:38433): tablet b921a9a719184abcba8006aaf14735d8 (table test-table [id=be7e984ca62247d6a85861aad523994e]) successfully deleted
I20260806 21:14:23.206743 7049 ts_tablet_manager.cc:1905] T b921a9a719184abcba8006aaf14735d8 P 07b6bdd51f164475a3477bad2ca26fed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:14:23.208148 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 6575
W20260806 21:14:23.208267 6239 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.178.1:36213 (error 108)
I20260806 21:14:23.217448 7014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:23.216971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:36213 (local address 127.0.0.1:35818)
0806 21:14:23.217216 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:23.217311 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.1:36213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150}
W20260806 21:14:23.217898 6239 catalog_manager.cc:4584] TS 5269f8bc3f5445f9af84a55eca845099 (127.16.178.1:36213): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b921a9a719184abcba8006aaf14735d8: Network error: Client connection negotiation failed: client connection to 127.16.178.1:36213: connect: Connection refused (error 111)
I20260806 21:14:23.218617 7049 ts_tablet_manager.cc:1918] T b921a9a719184abcba8006aaf14735d8 P 07b6bdd51f164475a3477bad2ca26fed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:14:23.218762 7049 log.cc:1196] T b921a9a719184abcba8006aaf14735d8 P 07b6bdd51f164475a3477bad2ca26fed: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-2/wal/wals/b921a9a719184abcba8006aaf14735d8
I20260806 21:14:23.219126 7049 ts_tablet_manager.cc:1939] T b921a9a719184abcba8006aaf14735d8 P 07b6bdd51f164475a3477bad2ca26fed: Deleting consensus metadata
I20260806 21:14:23.220598 6239 catalog_manager.cc:4841] TS 07b6bdd51f164475a3477bad2ca26fed (127.16.178.3:33729): tablet b921a9a719184abcba8006aaf14735d8 (table test-table [id=be7e984ca62247d6a85861aad523994e]) successfully deleted
I20260806 21:14:23.222183 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 6793
I20260806 21:14:23.239239 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 6213
I20260806 21:14:23.256531 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:35423
--webserver_interface=127.16.178.62
--webserver_port=42461
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:46639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:35423
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:14:23.376598 7051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:14:23.376925 7051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:14:23.376972 7051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:14:23.381085 7051 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:14:23.381178 7051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:14:23.381201 7051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:14:23.381220 7051 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:14:23.381238 7051 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:14:23.386929 7051 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:46639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:35423
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:35423
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.178.62
--webserver_port=42461
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:14:23.388164 7051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:14:23.389251 7051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:14:23.405689 7066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:14:23.405681 7065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:14:23.405754 7051 server_base.cc:1034] running on GCE node
W20260806 21:14:23.405681 7068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:14:23.406750 7051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:14:23.408269 7051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:14:23.409492 7051 hybrid_clock.cc:648] HybridClock initialized: now 1786050863409459 us; error 57 us; skew 500 ppm
Aug 06 21:14:23 dist-test-slave-lgr5 krb5kdc[6191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050863, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:14:23.412792 7051 init.cc:377] Logged in from keytab as kudu/127.16.178.62@KRBTEST.COM (short username kudu)
I20260806 21:14:23.413983 7051 webserver.cc:460] Webserver started at http://127.16.178.62:42461/ using document root <none> and password file <none>
I20260806 21:14:23.414660 7051 fs_manager.cc:359] Metadata directory not provided
I20260806 21:14:23.414753 7051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:14:23.419416 7051 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:14:23.422088 7079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:23.423488 7051 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:14:23.423635 7051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "b5b9bac96afb4727a08a2d7f92db68ea"
format_stamp: "Formatted at 2026-08-06 21:14:22 on dist-test-slave-lgr5"
server_key: "0046ffbcfdd50ddfd0180b4c19f74b1b"
server_key_iv: "c9d1f2ed017a450a5c39e8ea9905dbff"
server_key_version: "encryptionkey@0"
I20260806 21:14:23.424373 7051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:14:23.437106 7051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:14:23.440254 7051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:14:23.440461 7051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:14:23.447727 7051 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.62:35423
I20260806 21:14:23.448073 7155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.62:35423 every 8 connection(s)
I20260806 21:14:23.448944 7051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:14:23.453438 7156 sys_catalog.cc:263] Verifying existing consensus state
I20260806 21:14:23.454284 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 7051
I20260806 21:14:23.454735 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.1:36213
--local_ip_for_outbound_sockets=127.16.178.1
--tserver_master_addrs=127.16.178.62:35423
--webserver_port=35361
--webserver_interface=127.16.178.1
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab
--principal=kudu/127.16.178.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:46639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 21:14:23.455206 7156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea: Bootstrap starting.
I20260806 21:14:23.472671 7156 log.cc:824] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea: Log is configured to *not* fsync() on all Append() calls
I20260806 21:14:23.481638 7156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 21:14:23.482005 7156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea: Bootstrap complete.
I20260806 21:14:23.486245 7156 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5b9bac96afb4727a08a2d7f92db68ea" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 35423 } }
I20260806 21:14:23.486698 7156 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5b9bac96afb4727a08a2d7f92db68ea, State: Initialized, Role: FOLLOWER
I20260806 21:14:23.487248 7156 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5b9bac96afb4727a08a2d7f92db68ea" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 35423 } }
I20260806 21:14:23.487396 7156 raft_consensus.cc:388] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:14:23.487478 7156 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:14:23.487581 7156 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [term 1 FOLLOWER]: Advancing to term 2
I20260806 21:14:23.489825 7156 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5b9bac96afb4727a08a2d7f92db68ea" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 35423 } }
I20260806 21:14:23.490239 7156 leader_election.cc:302] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b5b9bac96afb4727a08a2d7f92db68ea; no voters:
I20260806 21:14:23.490605 7156 leader_election.cc:288] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 21:14:23.490760 7168 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [term 2 FOLLOWER]: Leader election won for term 2
I20260806 21:14:23.491052 7168 raft_consensus.cc:686] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [term 2 LEADER]: Becoming Leader. State: Replica: b5b9bac96afb4727a08a2d7f92db68ea, State: Running, Role: LEADER
I20260806 21:14:23.491415 7168 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5b9bac96afb4727a08a2d7f92db68ea" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 35423 } }
I20260806 21:14:23.492003 7156 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:14:23.493037 7169 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b5b9bac96afb4727a08a2d7f92db68ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5b9bac96afb4727a08a2d7f92db68ea" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 35423 } } }
I20260806 21:14:23.493176 7169 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [sys.catalog]: This master's current role is: LEADER
I20260806 21:14:23.493803 7172 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5b9bac96afb4727a08a2d7f92db68ea. Latest consensus state: current_term: 2 leader_uuid: "b5b9bac96afb4727a08a2d7f92db68ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5b9bac96afb4727a08a2d7f92db68ea" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 35423 } } }
I20260806 21:14:23.493907 7172 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [sys.catalog]: This master's current role is: LEADER
I20260806 21:14:23.494628 7179 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:14:23.499755 7179 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:14:23.500509 7179 catalog_manager.cc:1259] Loaded cluster ID: e759654df3f34f68be7a171971ffed5f
I20260806 21:14:23.500581 7179 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:14:23.502101 7179 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260806 21:14:23.506812 7179 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260806 21:14:23.513065 7179 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:14:23.514055 7179 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea: Loaded TSK: 0
I20260806 21:14:23.515569 7179 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 21:14:23.577757 7164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:14:23.578074 7164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:14:23.578141 7164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:14:23.582530 7164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:14:23.582638 7164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:14:23.582728 7164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.1
I20260806 21:14:23.588825 7164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:46639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab
--principal=kudu/127.16.178.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.1:36213
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.178.1
--webserver_port=35361
--tserver_master_addrs=127.16.178.62:35423
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.1
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:14:23.590070 7164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:14:23.591342 7164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:14:23.602550 7198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:14:23.602702 7164 server_base.cc:1034] running on GCE node
W20260806 21:14:23.602571 7203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:14:23.602955 7199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:14:23.603502 7164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:14:23.604373 7164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:14:23.605568 7164 hybrid_clock.cc:648] HybridClock initialized: now 1786050863605545 us; error 38 us; skew 500 ppm
Aug 06 21:14:23 dist-test-slave-lgr5 krb5kdc[6191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050863, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:14:23.608976 7164 init.cc:377] Logged in from keytab as kudu/127.16.178.1@KRBTEST.COM (short username kudu)
I20260806 21:14:23.610188 7164 webserver.cc:460] Webserver started at http://127.16.178.1:35361/ using document root <none> and password file <none>
I20260806 21:14:23.610852 7164 fs_manager.cc:359] Metadata directory not provided
I20260806 21:14:23.610957 7164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:14:23.615764 7164 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:14:23.618515 7210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:23.620009 7164 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:14:23.620184 7164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-0/wal
uuid: "5269f8bc3f5445f9af84a55eca845099"
format_stamp: "Formatted at 2026-08-06 21:14:22 on dist-test-slave-lgr5"
server_key: "7cd9a778070dd6249d86bd1408a5b3ba"
server_key_iv: "0066c2214e265ece377d8d83b9a17891"
server_key_version: "encryptionkey@0"
I20260806 21:14:23.620826 7164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:14:23.639011 7164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:14:23.642356 7164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:14:23.642540 7164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:14:23.643162 7164 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:14:23.644357 7220 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 21:14:23.648697 7164 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 21:14:23.648770 7164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260806 21:14:23.648841 7164 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 21:14:23.650542 7164 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 21:14:23.650588 7164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260806 21:14:23.650897 7220 tablet_bootstrap.cc:492] T b921a9a719184abcba8006aaf14735d8 P 5269f8bc3f5445f9af84a55eca845099: Bootstrap starting.
I20260806 21:14:23.660096 7164 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.1:36213
I20260806 21:14:23.660230 7392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.1:36213 every 8 connection(s)
I20260806 21:14:23.661406 7164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
I20260806 21:14:23.662503 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 7164
I20260806 21:14:23.662890 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.2:38433
--local_ip_for_outbound_sockets=127.16.178.2
--tserver_master_addrs=127.16.178.62:35423
--webserver_port=42233
--webserver_interface=127.16.178.2
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab
--principal=kudu/127.16.178.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:46639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 21:14:23 dist-test-slave-lgr5 krb5kdc[6191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050863, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:14:23.672618 7220 log.cc:824] T b921a9a719184abcba8006aaf14735d8 P 5269f8bc3f5445f9af84a55eca845099: Log is configured to *not* fsync() on all Append() calls
I20260806 21:14:23.678557 7396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:23.663777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:57411 (local address 127.16.178.62:35423)
0806 21:14:23.664310 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:23.664331 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:23.665784 (+ 1453us) server_negotiation.cc:407] Connection header received
0806 21:14:23.666012 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:23.666021 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:23.666306 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:23.666614 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:14:23.667774 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:23.668851 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:23.670004 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:23.670476 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:23.670630 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:23.673207 (+ 2577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:14:23.673213 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:14:23.673226 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:14:23.673258 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:14:23.675121 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:23.675683 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:23.675689 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:23.675694 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:23.675825 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:23.676180 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:23.676184 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:23.676186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:23.676595 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:14:23.676732 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:23.677331 (+ 599us) server_negotiation.cc:299] Negotiation successful
0806 21:14:23.677616 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":122,"threads_started":1}
I20260806 21:14:23.678583 7397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:23.663775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.1:57411)
0806 21:14:23.664638 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:23.664706 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0806 21:14:23.665540 (+ 834us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:14:23.666699 (+ 1159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:14:23.666707 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:14:23.667140 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:14:23.667599 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:23.667615 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:23.669010 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:23.669013 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:23.669872 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:23.669882 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:23.670623 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:23.670627 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:23.670809 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:23.671410 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:14:23.671429 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:14:23.672996 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:14:23.675264 (+ 2268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:23.675272 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:23.675286 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:23.675561 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:23.675920 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:23.675923 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:23.675925 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:23.676066 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:23.676719 (+ 653us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:14:23.676727 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:14:23.676999 (+ 272us) client_negotiation.cc:769] Sending connection context
0806 21:14:23.677171 (+ 172us) client_negotiation.cc:240] Negotiation successful
0806 21:14:23.677577 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":174,"threads_started":1}
I20260806 21:14:23.681236 7394 heartbeater.cc:347] Connected to a master server at 127.16.178.62:35423
I20260806 21:14:23.681495 7394 heartbeater.cc:464] Registering TS with master...
I20260806 21:14:23.682003 7394 heartbeater.cc:511] Master 127.16.178.62:35423 requested a full tablet report, sending...
I20260806 21:14:23.682948 7220 tablet_bootstrap.cc:492] T b921a9a719184abcba8006aaf14735d8 P 5269f8bc3f5445f9af84a55eca845099: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 21:14:23.683370 7220 tablet_bootstrap.cc:492] T b921a9a719184abcba8006aaf14735d8 P 5269f8bc3f5445f9af84a55eca845099: Bootstrap complete.
I20260806 21:14:23.684412 7105 ts_manager.cc:194] Registered new tserver with Master: 5269f8bc3f5445f9af84a55eca845099 (127.16.178.1:36213)
I20260806 21:14:23.684725 7220 ts_tablet_manager.cc:1397] T b921a9a719184abcba8006aaf14735d8 P 5269f8bc3f5445f9af84a55eca845099: Time spent bootstrapping tablet: real 0.034s user 0.017s sys 0.014s
I20260806 21:14:23.687819 7105 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.1@KRBTEST.COM'} at 127.16.178.1:57411
I20260806 21:14:23.687891 7220 raft_consensus.cc:348] T b921a9a719184abcba8006aaf14735d8 P 5269f8bc3f5445f9af84a55eca845099 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5269f8bc3f5445f9af84a55eca845099" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36213 } } peers { permanent_uuid: "e2779b8373194b8ba7a735b2c5c20c15" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38433 } } peers { permanent_uuid: "07b6bdd51f164475a3477bad2ca26fed" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33729 } }
I20260806 21:14:23.688460 7220 raft_consensus.cc:729] T b921a9a719184abcba8006aaf14735d8 P 5269f8bc3f5445f9af84a55eca845099 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5269f8bc3f5445f9af84a55eca845099, State: Initialized, Role: FOLLOWER
I20260806 21:14:23.688961 7220 consensus_queue.cc:260] T b921a9a719184abcba8006aaf14735d8 P 5269f8bc3f5445f9af84a55eca845099 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5269f8bc3f5445f9af84a55eca845099" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36213 } } peers { permanent_uuid: "e2779b8373194b8ba7a735b2c5c20c15" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38433 } } peers { permanent_uuid: "07b6bdd51f164475a3477bad2ca26fed" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33729 } }
I20260806 21:14:23.689988 7220 ts_tablet_manager.cc:1428] T b921a9a719184abcba8006aaf14735d8 P 5269f8bc3f5445f9af84a55eca845099: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260806 21:14:23.690742 7394 heartbeater.cc:503] Master 127.16.178.62:35423 was elected leader, sending a full tablet report...
I20260806 21:14:23.694437 7407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:23.687960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35828 (local address 127.16.178.1:36213)
0806 21:14:23.688627 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:23.688642 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:23.689255 (+ 613us) server_negotiation.cc:407] Connection header received
0806 21:14:23.689544 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:23.689549 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:23.689890 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:23.690034 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:14:23.690539 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:23.691477 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:23.693144 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:23.693782 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:23.693877 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:23.694023 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:23.694196 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 21:14:23.694288 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":406,"thread_start_us":143,"threads_started":1}
I20260806 21:14:23.694428 7402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:23.687203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:36213 (local address 127.0.0.1:35828)
0806 21:14:23.688942 (+ 1739us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:23.688990 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 21:14:23.689339 (+ 349us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:14:23.690248 (+ 909us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:14:23.690253 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:14:23.690277 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:14:23.690398 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:23.690405 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:23.691576 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:23.691580 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:23.693029 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:23.693040 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:23.693890 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:23.693894 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:23.694019 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:23.694061 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 21:14:23.694157 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 21:14:23.694293 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1568,"thread_start_us":113,"threads_started":1}
I20260806 21:14:23.695537 7285 tablet_service.cc:1514] Processing DeleteTablet for tablet b921a9a719184abcba8006aaf14735d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:14:23 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:35828
I20260806 21:14:23.696612 7409 tablet_replica.cc:330] stopping tablet replica
I20260806 21:14:23.696908 7409 raft_consensus.cc:2229] T b921a9a719184abcba8006aaf14735d8 P 5269f8bc3f5445f9af84a55eca845099 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:14:23.697147 7409 raft_consensus.cc:2258] T b921a9a719184abcba8006aaf14735d8 P 5269f8bc3f5445f9af84a55eca845099 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:14:23.698203 7409 ts_tablet_manager.cc:1905] T b921a9a719184abcba8006aaf14735d8 P 5269f8bc3f5445f9af84a55eca845099: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:14:23.701879 7409 ts_tablet_manager.cc:1918] T b921a9a719184abcba8006aaf14735d8 P 5269f8bc3f5445f9af84a55eca845099: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:14:23.701984 7409 log.cc:1196] T b921a9a719184abcba8006aaf14735d8 P 5269f8bc3f5445f9af84a55eca845099: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-0/wal/wals/b921a9a719184abcba8006aaf14735d8
I20260806 21:14:23.702215 7409 ts_tablet_manager.cc:1939] T b921a9a719184abcba8006aaf14735d8 P 5269f8bc3f5445f9af84a55eca845099: Deleting consensus metadata
I20260806 21:14:23.704069 7083 catalog_manager.cc:4841] TS 5269f8bc3f5445f9af84a55eca845099 (127.16.178.1:36213): tablet b921a9a719184abcba8006aaf14735d8 (table test-table [id=be7e984ca62247d6a85861aad523994e]) successfully deleted
W20260806 21:14:23.790856 7398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:14:23.791152 7398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:14:23.791198 7398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:14:23.795514 7398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:14:23.795588 7398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:14:23.795658 7398 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.2
I20260806 21:14:23.801043 7398 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:46639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab
--principal=kudu/127.16.178.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.2:38433
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.178.2
--webserver_port=42233
--tserver_master_addrs=127.16.178.62:35423
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.2
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:14:23.802186 7398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:14:23.803282 7398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:14:23.813174 7418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:14:23.813220 7398 server_base.cc:1034] running on GCE node
W20260806 21:14:23.813184 7421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:14:23.813179 7417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:14:23.813920 7398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:14:23.814585 7398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:14:23.815763 7398 hybrid_clock.cc:648] HybridClock initialized: now 1786050863815746 us; error 29 us; skew 500 ppm
Aug 06 21:14:23 dist-test-slave-lgr5 krb5kdc[6191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050863, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:14:23.818691 7398 init.cc:377] Logged in from keytab as kudu/127.16.178.2@KRBTEST.COM (short username kudu)
I20260806 21:14:23.819657 7398 webserver.cc:460] Webserver started at http://127.16.178.2:42233/ using document root <none> and password file <none>
I20260806 21:14:23.820287 7398 fs_manager.cc:359] Metadata directory not provided
I20260806 21:14:23.820387 7398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:14:23.824446 7398 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 21:14:23.826609 7436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:23.827896 7398 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 21:14:23.828011 7398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-1/wal
uuid: "e2779b8373194b8ba7a735b2c5c20c15"
format_stamp: "Formatted at 2026-08-06 21:14:22 on dist-test-slave-lgr5"
server_key: "615f7c32a01ce888513a082d28ad21ff"
server_key_iv: "3804dc8dbd34e394e35749f1a656045e"
server_key_version: "encryptionkey@0"
I20260806 21:14:23.828604 7398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:14:23.847903 7398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:14:23.851058 7398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:14:23.851222 7398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:14:23.851775 7398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:14:23.852670 7398 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:14:23.852720 7398 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:23.852761 7398 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:14:23.852774 7398 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:23.861281 7398 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.2:38433
I20260806 21:14:23.861320 7615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.2:38433 every 8 connection(s)
I20260806 21:14:23.862392 7398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
I20260806 21:14:23.862898 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 7398
I20260806 21:14:23.863360 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.3:33729
--local_ip_for_outbound_sockets=127.16.178.3
--tserver_master_addrs=127.16.178.62:35423
--webserver_port=45195
--webserver_interface=127.16.178.3
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab
--principal=kudu/127.16.178.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:46639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 21:14:23 dist-test-slave-lgr5 krb5kdc[6191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050863, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.2@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:14:23.877254 7396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:23.864510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:36187 (local address 127.16.178.62:35423)
0806 21:14:23.864678 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:23.864684 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:23.865733 (+ 1049us) server_negotiation.cc:407] Connection header received
0806 21:14:23.867155 (+ 1422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:23.867159 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:23.867230 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:23.867305 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:14:23.868592 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:23.869122 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:23.870294 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:23.870548 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:23.870646 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:23.873443 (+ 2797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:14:23.873448 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:14:23.873451 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:14:23.873487 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:14:23.875113 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:23.875665 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:23.875668 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:23.875670 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:23.875758 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:23.876057 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:23.876060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:23.876062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:23.876321 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:14:23.876414 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:23.876925 (+ 511us) server_negotiation.cc:299] Negotiation successful
0806 21:14:23.877072 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 21:14:23.878095 7624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:23.864996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.2:36187)
0806 21:14:23.865569 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:23.865615 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 21:14:23.866933 (+ 1318us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:14:23.867502 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:14:23.867509 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:14:23.867880 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:14:23.868440 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:23.868455 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:23.869254 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:23.869257 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:23.870169 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:23.870180 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:23.870707 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:23.870712 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:23.870886 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:23.871455 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:14:23.871472 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:14:23.873210 (+ 1738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:14:23.875254 (+ 2044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:23.875260 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:23.875274 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:23.875574 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:23.875837 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:23.875839 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:23.875840 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:23.875986 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:23.876422 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:14:23.876428 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:14:23.876699 (+ 271us) client_negotiation.cc:769] Sending connection context
0806 21:14:23.876869 (+ 170us) client_negotiation.cc:240] Negotiation successful
0806 21:14:23.877152 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":163,"threads_started":1}
I20260806 21:14:23.879433 7616 heartbeater.cc:347] Connected to a master server at 127.16.178.62:35423
I20260806 21:14:23.879684 7616 heartbeater.cc:464] Registering TS with master...
I20260806 21:14:23.880254 7616 heartbeater.cc:511] Master 127.16.178.62:35423 requested a full tablet report, sending...
I20260806 21:14:23.881402 7105 ts_manager.cc:194] Registered new tserver with Master: e2779b8373194b8ba7a735b2c5c20c15 (127.16.178.2:38433)
I20260806 21:14:23.881989 7105 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.2@KRBTEST.COM'} at 127.16.178.2:36187
W20260806 21:14:23.984786 7622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:14:23.985108 7622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:14:23.985158 7622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:14:23.989544 7622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:14:23.989639 7622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:14:23.989703 7622 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.3
I20260806 21:14:23.996156 7622 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:46639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab
--principal=kudu/127.16.178.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.3:33729
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.178.3
--webserver_port=45195
--tserver_master_addrs=127.16.178.62:35423
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7622
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.3
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:14:23.998209 7622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:14:23.999811 7622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:14:24.015285 7632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:14:24.015276 7635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:14:24.016940 7630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:14:24.017897 7622 server_base.cc:1034] running on GCE node
I20260806 21:14:24.018687 7622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:14:24.019743 7622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:14:24.020947 7622 hybrid_clock.cc:648] HybridClock initialized: now 1786050864020917 us; error 79 us; skew 500 ppm
Aug 06 21:14:24 dist-test-slave-lgr5 krb5kdc[6191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050864, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:14:24.024876 7622 init.cc:377] Logged in from keytab as kudu/127.16.178.3@KRBTEST.COM (short username kudu)
I20260806 21:14:24.030555 7622 webserver.cc:460] Webserver started at http://127.16.178.3:45195/ using document root <none> and password file <none>
I20260806 21:14:24.031332 7622 fs_manager.cc:359] Metadata directory not provided
I20260806 21:14:24.031443 7622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:14:24.037776 7622 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 21:14:24.041009 7648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:24.042840 7622 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 21:14:24.042999 7622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-2/wal
uuid: "07b6bdd51f164475a3477bad2ca26fed"
format_stamp: "Formatted at 2026-08-06 21:14:22 on dist-test-slave-lgr5"
server_key: "a09fc69c01e6c42d66c3a02942325f84"
server_key_iv: "5da7c4883bf97482abb5b8fb88741fa9"
server_key_version: "encryptionkey@0"
I20260806 21:14:24.043879 7622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:14:24.061906 7622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:14:24.065492 7622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:14:24.065697 7622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:14:24.066355 7622 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:14:24.067302 7622 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:14:24.067360 7622 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:24.067408 7622 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:14:24.067432 7622 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:24.078117 7622 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.3:33729
I20260806 21:14:24.078164 7812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.3:33729 every 8 connection(s)
I20260806 21:14:24.079262 7622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
I20260806 21:14:24.081835 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 7622
Aug 06 21:14:24 dist-test-slave-lgr5 krb5kdc[6191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050864, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.3@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:14:24.091576 7819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:24.082636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48494 (local address 127.16.178.62:35423)
0806 21:14:24.082948 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:24.082953 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:24.082967 (+ 14us) server_negotiation.cc:407] Connection header received
0806 21:14:24.083134 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:24.083138 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:24.083187 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:24.083266 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:14:24.084132 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:24.084719 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:24.085621 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:24.085913 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:24.086006 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:24.087429 (+ 1423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:14:24.087433 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:14:24.087435 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:14:24.087461 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:14:24.089007 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:24.090010 (+ 1003us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:24.090013 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:24.090015 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:24.090114 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:24.090429 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:24.090433 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:24.090435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:24.090682 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:14:24.090877 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:24.091183 (+ 306us) server_negotiation.cc:299] Negotiation successful
0806 21:14:24.091337 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":244,"thread_start_us":101,"threads_started":1}
I20260806 21:14:24.094718 7396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:24.081330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:56881 (local address 127.16.178.62:35423)
0806 21:14:24.081523 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:24.081529 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:24.082515 (+ 986us) server_negotiation.cc:407] Connection header received
0806 21:14:24.084157 (+ 1642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:24.084160 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:24.084218 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:24.084319 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:14:24.085628 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:24.086168 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:24.087380 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:24.087606 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:24.087707 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:24.090254 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:14:24.090257 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:14:24.090259 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:14:24.090345 (+ 86us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:14:24.092153 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:24.092898 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:24.092901 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:24.092902 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:24.092994 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:24.093439 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:24.093442 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:24.093443 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:24.093679 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:14:24.093772 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:24.094408 (+ 636us) server_negotiation.cc:299] Negotiation successful
0806 21:14:24.094542 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":63}
I20260806 21:14:24.095634 7816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:24.081773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.3:56881)
0806 21:14:24.082342 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:24.082391 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 21:14:24.083898 (+ 1507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:14:24.084518 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:14:24.084525 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:14:24.084945 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:14:24.085469 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:24.085484 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:24.086315 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:24.086318 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:24.087240 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:24.087252 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:24.087781 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:24.087786 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:24.087965 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:24.088478 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:14:24.088496 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:14:24.090083 (+ 1587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:14:24.092273 (+ 2190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:24.092281 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:24.092302 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:24.092671 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:24.093099 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:24.093102 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:24.093103 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:24.093322 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:24.093784 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:14:24.093792 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:14:24.094147 (+ 355us) client_negotiation.cc:769] Sending connection context
0806 21:14:24.094334 (+ 187us) client_negotiation.cc:240] Negotiation successful
0806 21:14:24.094718 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":145,"threads_started":1}
I20260806 21:14:24.096953 7814 heartbeater.cc:347] Connected to a master server at 127.16.178.62:35423
I20260806 21:14:24.097224 7814 heartbeater.cc:464] Registering TS with master...
I20260806 21:14:24.097790 7814 heartbeater.cc:511] Master 127.16.178.62:35423 requested a full tablet report, sending...
I20260806 21:14:24.099037 7105 ts_manager.cc:194] Registered new tserver with Master: 07b6bdd51f164475a3477bad2ca26fed (127.16.178.3:33729)
I20260806 21:14:24.099694 7105 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.3@KRBTEST.COM'} at 127.16.178.3:56881
I20260806 21:14:24.106128 17096 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 21:14:24.110916 7396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:24.106823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48496 (local address 127.16.178.62:35423)
0806 21:14:24.106996 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:24.107001 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:24.107165 (+ 164us) server_negotiation.cc:407] Connection header received
0806 21:14:24.107402 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:24.107405 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:24.107538 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:24.107644 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:14:24.107856 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:24.108547 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:24.109387 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:24.109625 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:24.109706 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:24.110005 (+ 299us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:14:24.110497 (+ 492us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:14:24.110616 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:24.110700 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 21:14:24.110755 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 21:14:24.111639 7105 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:48496:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:14:24.115203 7105 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:14:24.124193 7285 tablet_service.cc:1467] Processing CreateTablet for tablet 613ce14121fa4d4fb70f6e72a08f0814 (DEFAULT_TABLE table=test-table [id=ca571752d13a4c4ca64289aaf95f6209]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:14:24.125015 7285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 613ce14121fa4d4fb70f6e72a08f0814. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:14:24.128463 7220 tablet_bootstrap.cc:492] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099: Bootstrap starting.
I20260806 21:14:24.129280 7402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:24.122922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.2:38433 (local address 127.0.0.1:53368)
0806 21:14:24.123077 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:24.123093 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:14:24.123189 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:14:24.124575 (+ 1386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:14:24.124579 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:14:24.124600 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:14:24.124747 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:24.124754 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:24.126369 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:24.126372 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:24.127515 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:24.127531 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:24.128807 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:24.128811 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:24.128903 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:24.128933 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 21:14:24.129032 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 21:14:24.129132 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79,"mutex_wait_us":30}
I20260806 21:14:24.129530 7825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:24.123032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.3:33729 (local address 127.0.0.1:56844)
0806 21:14:24.123413 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:24.123427 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:14:24.123516 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:14:24.124748 (+ 1232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:14:24.124751 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:14:24.124768 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:14:24.124871 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:24.124885 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:24.126292 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:24.126296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:24.127543 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:24.127556 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:24.129066 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:24.129070 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:24.129186 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:24.129211 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 21:14:24.129281 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 21:14:24.129371 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"mutex_wait_us":25,"thread_start_us":133,"threads_started":1}
I20260806 21:14:24.129561 7220 tablet_bootstrap.cc:654] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099: Neither blocks nor log segments found. Creating new log.
I20260806 21:14:24.129734 7828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:24.123354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53368 (local address 127.16.178.2:38433)
0806 21:14:24.124037 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:24.124050 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:24.124099 (+ 49us) server_negotiation.cc:407] Connection header received
0806 21:14:24.124188 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:24.124193 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:24.124410 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:24.124946 (+ 536us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:14:24.125129 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:24.126172 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:24.127677 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:24.128626 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:24.128779 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:24.128968 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:24.129467 (+ 499us) server_negotiation.cc:299] Negotiation successful
0806 21:14:24.129587 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":180,"threads_started":1}
I20260806 21:14:24.129971 7829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:24.123764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56844 (local address 127.16.178.3:33729)
0806 21:14:24.124190 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:24.124206 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:24.124232 (+ 26us) server_negotiation.cc:407] Connection header received
0806 21:14:24.124315 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:24.124320 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:24.124541 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:24.124721 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:14:24.124980 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:24.126113 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:24.127702 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:24.128773 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:24.128879 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:24.129009 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:24.129708 (+ 699us) server_negotiation.cc:299] Negotiation successful
0806 21:14:24.129821 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":172,"threads_started":1}
I20260806 21:14:24.131258 7220 tablet_bootstrap.cc:492] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099: No bootstrap required, opened a new log
I20260806 21:14:24.131364 7220 ts_tablet_manager.cc:1397] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260806 21:14:24.131841 7220 raft_consensus.cc:348] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2779b8373194b8ba7a735b2c5c20c15" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38433 } } peers { permanent_uuid: "07b6bdd51f164475a3477bad2ca26fed" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33729 } } peers { permanent_uuid: "5269f8bc3f5445f9af84a55eca845099" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36213 } }
I20260806 21:14:24.131966 7220 raft_consensus.cc:374] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:14:24.132005 7220 raft_consensus.cc:729] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5269f8bc3f5445f9af84a55eca845099, State: Initialized, Role: FOLLOWER
I20260806 21:14:24.132191 7220 consensus_queue.cc:260] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2779b8373194b8ba7a735b2c5c20c15" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38433 } } peers { permanent_uuid: "07b6bdd51f164475a3477bad2ca26fed" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33729 } } peers { permanent_uuid: "5269f8bc3f5445f9af84a55eca845099" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36213 } }
I20260806 21:14:24.132472 7220 ts_tablet_manager.cc:1428] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 21:14:24.133785 7514 tablet_service.cc:1467] Processing CreateTablet for tablet 613ce14121fa4d4fb70f6e72a08f0814 (DEFAULT_TABLE table=test-table [id=ca571752d13a4c4ca64289aaf95f6209]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:14:24.133786 7717 tablet_service.cc:1467] Processing CreateTablet for tablet 613ce14121fa4d4fb70f6e72a08f0814 (DEFAULT_TABLE table=test-table [id=ca571752d13a4c4ca64289aaf95f6209]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:14:24.134963 7717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 613ce14121fa4d4fb70f6e72a08f0814. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:14:24.134955 7514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 613ce14121fa4d4fb70f6e72a08f0814. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:14:24.143098 7832 tablet_bootstrap.cc:492] T 613ce14121fa4d4fb70f6e72a08f0814 P e2779b8373194b8ba7a735b2c5c20c15: Bootstrap starting.
I20260806 21:14:24.144138 7834 tablet_bootstrap.cc:492] T 613ce14121fa4d4fb70f6e72a08f0814 P 07b6bdd51f164475a3477bad2ca26fed: Bootstrap starting.
I20260806 21:14:24.146210 7832 tablet_bootstrap.cc:654] T 613ce14121fa4d4fb70f6e72a08f0814 P e2779b8373194b8ba7a735b2c5c20c15: Neither blocks nor log segments found. Creating new log.
I20260806 21:14:24.146421 7834 tablet_bootstrap.cc:654] T 613ce14121fa4d4fb70f6e72a08f0814 P 07b6bdd51f164475a3477bad2ca26fed: Neither blocks nor log segments found. Creating new log.
I20260806 21:14:24.147073 7832 log.cc:824] T 613ce14121fa4d4fb70f6e72a08f0814 P e2779b8373194b8ba7a735b2c5c20c15: Log is configured to *not* fsync() on all Append() calls
I20260806 21:14:24.147135 7834 log.cc:824] T 613ce14121fa4d4fb70f6e72a08f0814 P 07b6bdd51f164475a3477bad2ca26fed: Log is configured to *not* fsync() on all Append() calls
I20260806 21:14:24.149319 7834 tablet_bootstrap.cc:492] T 613ce14121fa4d4fb70f6e72a08f0814 P 07b6bdd51f164475a3477bad2ca26fed: No bootstrap required, opened a new log
I20260806 21:14:24.149326 7832 tablet_bootstrap.cc:492] T 613ce14121fa4d4fb70f6e72a08f0814 P e2779b8373194b8ba7a735b2c5c20c15: No bootstrap required, opened a new log
I20260806 21:14:24.149533 7834 ts_tablet_manager.cc:1397] T 613ce14121fa4d4fb70f6e72a08f0814 P 07b6bdd51f164475a3477bad2ca26fed: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 21:14:24.149540 7832 ts_tablet_manager.cc:1397] T 613ce14121fa4d4fb70f6e72a08f0814 P e2779b8373194b8ba7a735b2c5c20c15: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 21:14:24.152714 7834 raft_consensus.cc:348] T 613ce14121fa4d4fb70f6e72a08f0814 P 07b6bdd51f164475a3477bad2ca26fed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2779b8373194b8ba7a735b2c5c20c15" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38433 } } peers { permanent_uuid: "07b6bdd51f164475a3477bad2ca26fed" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33729 } } peers { permanent_uuid: "5269f8bc3f5445f9af84a55eca845099" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36213 } }
I20260806 21:14:24.152709 7832 raft_consensus.cc:348] T 613ce14121fa4d4fb70f6e72a08f0814 P e2779b8373194b8ba7a735b2c5c20c15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2779b8373194b8ba7a735b2c5c20c15" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38433 } } peers { permanent_uuid: "07b6bdd51f164475a3477bad2ca26fed" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33729 } } peers { permanent_uuid: "5269f8bc3f5445f9af84a55eca845099" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36213 } }
I20260806 21:14:24.152966 7834 raft_consensus.cc:374] T 613ce14121fa4d4fb70f6e72a08f0814 P 07b6bdd51f164475a3477bad2ca26fed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:14:24.152968 7832 raft_consensus.cc:374] T 613ce14121fa4d4fb70f6e72a08f0814 P e2779b8373194b8ba7a735b2c5c20c15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:14:24.153002 7834 raft_consensus.cc:729] T 613ce14121fa4d4fb70f6e72a08f0814 P 07b6bdd51f164475a3477bad2ca26fed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b6bdd51f164475a3477bad2ca26fed, State: Initialized, Role: FOLLOWER
I20260806 21:14:24.153023 7832 raft_consensus.cc:729] T 613ce14121fa4d4fb70f6e72a08f0814 P e2779b8373194b8ba7a735b2c5c20c15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2779b8373194b8ba7a735b2c5c20c15, State: Initialized, Role: FOLLOWER
I20260806 21:14:24.153435 7834 consensus_queue.cc:260] T 613ce14121fa4d4fb70f6e72a08f0814 P 07b6bdd51f164475a3477bad2ca26fed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2779b8373194b8ba7a735b2c5c20c15" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38433 } } peers { permanent_uuid: "07b6bdd51f164475a3477bad2ca26fed" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33729 } } peers { permanent_uuid: "5269f8bc3f5445f9af84a55eca845099" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36213 } }
I20260806 21:14:24.153436 7832 consensus_queue.cc:260] T 613ce14121fa4d4fb70f6e72a08f0814 P e2779b8373194b8ba7a735b2c5c20c15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2779b8373194b8ba7a735b2c5c20c15" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38433 } } peers { permanent_uuid: "07b6bdd51f164475a3477bad2ca26fed" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33729 } } peers { permanent_uuid: "5269f8bc3f5445f9af84a55eca845099" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36213 } }
I20260806 21:14:24.154150 7616 heartbeater.cc:503] Master 127.16.178.62:35423 was elected leader, sending a full tablet report...
I20260806 21:14:24.154150 7814 heartbeater.cc:503] Master 127.16.178.62:35423 was elected leader, sending a full tablet report...
I20260806 21:14:24.154542 7834 ts_tablet_manager.cc:1428] T 613ce14121fa4d4fb70f6e72a08f0814 P 07b6bdd51f164475a3477bad2ca26fed: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 21:14:24.154542 7832 ts_tablet_manager.cc:1428] T 613ce14121fa4d4fb70f6e72a08f0814 P e2779b8373194b8ba7a735b2c5c20c15: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260806 21:14:24.162840 7395 tablet.cc:2369] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 21:14:24.211782 7839 raft_consensus.cc:482] T 613ce14121fa4d4fb70f6e72a08f0814 P e2779b8373194b8ba7a735b2c5c20c15 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:14:24.212055 7839 raft_consensus.cc:504] T 613ce14121fa4d4fb70f6e72a08f0814 P e2779b8373194b8ba7a735b2c5c20c15 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2779b8373194b8ba7a735b2c5c20c15" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38433 } } peers { permanent_uuid: "07b6bdd51f164475a3477bad2ca26fed" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33729 } } peers { permanent_uuid: "5269f8bc3f5445f9af84a55eca845099" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36213 } }
I20260806 21:14:24.213339 7839 leader_election.cc:288] T 613ce14121fa4d4fb70f6e72a08f0814 P e2779b8373194b8ba7a735b2c5c20c15 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07b6bdd51f164475a3477bad2ca26fed (127.16.178.3:33729), 5269f8bc3f5445f9af84a55eca845099 (127.16.178.1:36213)
I20260806 21:14:24.217464 7624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:24.213638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.3:33729 (local address 127.16.178.2:34005)
0806 21:14:24.213818 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:24.213861 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 21:14:24.214026 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:14:24.214311 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:14:24.214314 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:14:24.214333 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:14:24.214461 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:24.214467 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:24.215378 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:24.215381 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:24.216452 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:24.216463 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:24.217112 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:24.217114 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:24.217197 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:24.217219 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 21:14:24.217275 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 21:14:24.217340 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 21:14:24.217502 7829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:24.213786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:34005 (local address 127.16.178.3:33729)
0806 21:14:24.213943 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:24.213947 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:24.213962 (+ 15us) server_negotiation.cc:407] Connection header received
0806 21:14:24.214156 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:24.214159 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:24.214210 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:24.214280 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:14:24.214574 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:24.215266 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:24.216571 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:24.216996 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:24.217117 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:24.217174 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:24.217300 (+ 126us) server_negotiation.cc:299] Negotiation successful
0806 21:14:24.217368 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 21:14:24.218166 7840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:24.214040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:36213 (local address 127.16.178.2:44245)
0806 21:14:24.214499 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:24.214515 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:14:24.214631 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:14:24.214978 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:14:24.214982 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:14:24.215003 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:14:24.215132 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:24.215137 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:24.215869 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:24.215872 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:24.216880 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:24.216888 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:24.217782 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:24.217786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:24.217872 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:24.217890 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 21:14:24.217954 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 21:14:24.218025 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":126,"threads_started":1}
I20260806 21:14:24.218178 7841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:24.214440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:44245 (local address 127.16.178.1:36213)
0806 21:14:24.214716 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:24.214720 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:24.214733 (+ 13us) server_negotiation.cc:407] Connection header received
0806 21:14:24.214786 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:24.214789 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:24.214835 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:24.214942 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:14:24.215236 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:24.215761 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:24.216997 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:24.217552 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:24.217670 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:24.217724 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:24.217988 (+ 264us) server_negotiation.cc:299] Negotiation successful
0806 21:14:24.218062 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":108,"threads_started":1}
I20260806 21:14:24.218225 7745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "613ce14121fa4d4fb70f6e72a08f0814" candidate_uuid: "e2779b8373194b8ba7a735b2c5c20c15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b6bdd51f164475a3477bad2ca26fed" is_pre_election: true
I20260806 21:14:24.218648 7745 raft_consensus.cc:2454] T 613ce14121fa4d4fb70f6e72a08f0814 P 07b6bdd51f164475a3477bad2ca26fed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2779b8373194b8ba7a735b2c5c20c15 in term 0.
I20260806 21:14:24.218958 7318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "613ce14121fa4d4fb70f6e72a08f0814" candidate_uuid: "e2779b8373194b8ba7a735b2c5c20c15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5269f8bc3f5445f9af84a55eca845099" is_pre_election: true
I20260806 21:14:24.219144 7437 leader_election.cc:302] T 613ce14121fa4d4fb70f6e72a08f0814 P e2779b8373194b8ba7a735b2c5c20c15 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b6bdd51f164475a3477bad2ca26fed, e2779b8373194b8ba7a735b2c5c20c15; no voters:
I20260806 21:14:24.219277 7318 raft_consensus.cc:2454] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2779b8373194b8ba7a735b2c5c20c15 in term 0.
I20260806 21:14:24.219457 7839 raft_consensus.cc:2781] T 613ce14121fa4d4fb70f6e72a08f0814 P e2779b8373194b8ba7a735b2c5c20c15 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:14:24.219542 7839 raft_consensus.cc:482] T 613ce14121fa4d4fb70f6e72a08f0814 P e2779b8373194b8ba7a735b2c5c20c15 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:14:24.219590 7839 raft_consensus.cc:3037] T 613ce14121fa4d4fb70f6e72a08f0814 P e2779b8373194b8ba7a735b2c5c20c15 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:14:24.220746 7839 raft_consensus.cc:504] T 613ce14121fa4d4fb70f6e72a08f0814 P e2779b8373194b8ba7a735b2c5c20c15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2779b8373194b8ba7a735b2c5c20c15" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38433 } } peers { permanent_uuid: "07b6bdd51f164475a3477bad2ca26fed" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33729 } } peers { permanent_uuid: "5269f8bc3f5445f9af84a55eca845099" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36213 } }
I20260806 21:14:24.221095 7839 leader_election.cc:288] T 613ce14121fa4d4fb70f6e72a08f0814 P e2779b8373194b8ba7a735b2c5c20c15 [CANDIDATE]: Term 1 election: Requested vote from peers 07b6bdd51f164475a3477bad2ca26fed (127.16.178.3:33729), 5269f8bc3f5445f9af84a55eca845099 (127.16.178.1:36213)
I20260806 21:14:24.221472 7745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "613ce14121fa4d4fb70f6e72a08f0814" candidate_uuid: "e2779b8373194b8ba7a735b2c5c20c15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b6bdd51f164475a3477bad2ca26fed"
I20260806 21:14:24.221558 7318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "613ce14121fa4d4fb70f6e72a08f0814" candidate_uuid: "e2779b8373194b8ba7a735b2c5c20c15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5269f8bc3f5445f9af84a55eca845099"
I20260806 21:14:24.221591 7745 raft_consensus.cc:3037] T 613ce14121fa4d4fb70f6e72a08f0814 P 07b6bdd51f164475a3477bad2ca26fed [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:14:24.221698 7318 raft_consensus.cc:3037] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:14:24.222652 7745 raft_consensus.cc:2454] T 613ce14121fa4d4fb70f6e72a08f0814 P 07b6bdd51f164475a3477bad2ca26fed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2779b8373194b8ba7a735b2c5c20c15 in term 1.
I20260806 21:14:24.222713 7318 raft_consensus.cc:2454] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2779b8373194b8ba7a735b2c5c20c15 in term 1.
I20260806 21:14:24.222944 7437 leader_election.cc:302] T 613ce14121fa4d4fb70f6e72a08f0814 P e2779b8373194b8ba7a735b2c5c20c15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b6bdd51f164475a3477bad2ca26fed, e2779b8373194b8ba7a735b2c5c20c15; no voters:
I20260806 21:14:24.223104 7839 raft_consensus.cc:2781] T 613ce14121fa4d4fb70f6e72a08f0814 P e2779b8373194b8ba7a735b2c5c20c15 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:14:24.223376 7839 raft_consensus.cc:686] T 613ce14121fa4d4fb70f6e72a08f0814 P e2779b8373194b8ba7a735b2c5c20c15 [term 1 LEADER]: Becoming Leader. State: Replica: e2779b8373194b8ba7a735b2c5c20c15, State: Running, Role: LEADER
I20260806 21:14:24.223773 7839 consensus_queue.cc:237] T 613ce14121fa4d4fb70f6e72a08f0814 P e2779b8373194b8ba7a735b2c5c20c15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2779b8373194b8ba7a735b2c5c20c15" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38433 } } peers { permanent_uuid: "07b6bdd51f164475a3477bad2ca26fed" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33729 } } peers { permanent_uuid: "5269f8bc3f5445f9af84a55eca845099" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36213 } }
I20260806 21:14:24.227608 7103 catalog_manager.cc:5495] T 613ce14121fa4d4fb70f6e72a08f0814 P e2779b8373194b8ba7a735b2c5c20c15 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2779b8373194b8ba7a735b2c5c20c15 (127.16.178.2). New cstate: current_term: 1 leader_uuid: "e2779b8373194b8ba7a735b2c5c20c15" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2779b8373194b8ba7a735b2c5c20c15" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07b6bdd51f164475a3477bad2ca26fed" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5269f8bc3f5445f9af84a55eca845099" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36213 } health_report { overall_health: UNKNOWN } } }
I20260806 21:14:24.240420 7828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:24.236864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53374 (local address 127.16.178.2:38433)
0806 21:14:24.237022 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:24.237026 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:24.237041 (+ 15us) server_negotiation.cc:407] Connection header received
0806 21:14:24.237123 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:24.237125 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:24.237187 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:24.237283 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:14:24.237592 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:24.238120 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:24.238919 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:24.239132 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:24.239210 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:24.239472 (+ 262us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:14:24.239991 (+ 519us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:14:24.240156 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:24.240212 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 21:14:24.240280 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 21:14:24.247824 7318 raft_consensus.cc:1261] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099 [term 1 FOLLOWER]: Refusing update from remote peer e2779b8373194b8ba7a735b2c5c20c15: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:14:24.247829 7745 raft_consensus.cc:1261] T 613ce14121fa4d4fb70f6e72a08f0814 P 07b6bdd51f164475a3477bad2ca26fed [term 1 FOLLOWER]: Refusing update from remote peer e2779b8373194b8ba7a735b2c5c20c15: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:14:24.248643 7839 consensus_queue.cc:1035] T 613ce14121fa4d4fb70f6e72a08f0814 P e2779b8373194b8ba7a735b2c5c20c15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5269f8bc3f5445f9af84a55eca845099" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:14:24.248896 7842 consensus_queue.cc:1035] T 613ce14121fa4d4fb70f6e72a08f0814 P e2779b8373194b8ba7a735b2c5c20c15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b6bdd51f164475a3477bad2ca26fed" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:14:24.254463 7846 mvcc.cc:205] Tried to move back new op lower bound from 7315664339949916160 to 7315664339863990272. Current Snapshot: MvccSnapshot[applied={T|T < 7315664339949916160}]
I20260806 21:14:24.255043 7847 mvcc.cc:205] Tried to move back new op lower bound from 7315664339949916160 to 7315664339863990272. Current Snapshot: MvccSnapshot[applied={T|T < 7315664339949916160}]
I20260806 21:14:24.265295 7103 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:48496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:14:24.265481 7103 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:48496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:14:24.267868 7103 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea: Sending DeleteTablet for 3 replicas of tablet 613ce14121fa4d4fb70f6e72a08f0814
I20260806 21:14:24.268646 7514 tablet_service.cc:1514] Processing DeleteTablet for tablet 613ce14121fa4d4fb70f6e72a08f0814 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:14:24 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:53368
I20260806 21:14:24.268730 7717 tablet_service.cc:1514] Processing DeleteTablet for tablet 613ce14121fa4d4fb70f6e72a08f0814 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:14:24 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:56844
I20260806 21:14:24.269171 7285 tablet_service.cc:1514] Processing DeleteTablet for tablet 613ce14121fa4d4fb70f6e72a08f0814 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:14:24 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:35828
I20260806 21:14:24.269229 7860 tablet_replica.cc:330] T 613ce14121fa4d4fb70f6e72a08f0814 P 07b6bdd51f164475a3477bad2ca26fed: stopping tablet replica
I20260806 21:14:24.269229 7858 tablet_replica.cc:330] T 613ce14121fa4d4fb70f6e72a08f0814 P e2779b8373194b8ba7a735b2c5c20c15: stopping tablet replica
I20260806 21:14:24.269585 7858 raft_consensus.cc:2229] T 613ce14121fa4d4fb70f6e72a08f0814 P e2779b8373194b8ba7a735b2c5c20c15 [term 1 LEADER]: Raft consensus shutting down.
I20260806 21:14:24.269584 7860 raft_consensus.cc:2229] T 613ce14121fa4d4fb70f6e72a08f0814 P 07b6bdd51f164475a3477bad2ca26fed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:14:24.269884 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 7164
I20260806 21:14:24.269948 7860 raft_consensus.cc:2258] T 613ce14121fa4d4fb70f6e72a08f0814 P 07b6bdd51f164475a3477bad2ca26fed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:14:24.270121 7858 raft_consensus.cc:2258] T 613ce14121fa4d4fb70f6e72a08f0814 P e2779b8373194b8ba7a735b2c5c20c15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:14:24.280136 7858 ts_tablet_manager.cc:1905] T 613ce14121fa4d4fb70f6e72a08f0814 P e2779b8373194b8ba7a735b2c5c20c15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:14:24.282886 7858 ts_tablet_manager.cc:1918] T 613ce14121fa4d4fb70f6e72a08f0814 P e2779b8373194b8ba7a735b2c5c20c15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:14:24.283092 7858 log.cc:1196] T 613ce14121fa4d4fb70f6e72a08f0814 P e2779b8373194b8ba7a735b2c5c20c15: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-1/wal/wals/613ce14121fa4d4fb70f6e72a08f0814
I20260806 21:14:24.283484 7858 ts_tablet_manager.cc:1939] T 613ce14121fa4d4fb70f6e72a08f0814 P e2779b8373194b8ba7a735b2c5c20c15: Deleting consensus metadata
I20260806 21:14:24.284950 7087 catalog_manager.cc:4841] TS e2779b8373194b8ba7a735b2c5c20c15 (127.16.178.2:38433): tablet 613ce14121fa4d4fb70f6e72a08f0814 (table test-table [id=ca571752d13a4c4ca64289aaf95f6209]) successfully deleted
I20260806 21:14:24.309461 7860 ts_tablet_manager.cc:1905] T 613ce14121fa4d4fb70f6e72a08f0814 P 07b6bdd51f164475a3477bad2ca26fed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:14:24.313097 7860 ts_tablet_manager.cc:1918] T 613ce14121fa4d4fb70f6e72a08f0814 P 07b6bdd51f164475a3477bad2ca26fed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:14:24.313344 7860 log.cc:1196] T 613ce14121fa4d4fb70f6e72a08f0814 P 07b6bdd51f164475a3477bad2ca26fed: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-2/wal/wals/613ce14121fa4d4fb70f6e72a08f0814
I20260806 21:14:24.313846 7860 ts_tablet_manager.cc:1939] T 613ce14121fa4d4fb70f6e72a08f0814 P 07b6bdd51f164475a3477bad2ca26fed: Deleting consensus metadata
I20260806 21:14:24.315666 7083 catalog_manager.cc:4841] TS 07b6bdd51f164475a3477bad2ca26fed (127.16.178.3:33729): tablet 613ce14121fa4d4fb70f6e72a08f0814 (table test-table [id=ca571752d13a4c4ca64289aaf95f6209]) successfully deleted
W20260806 21:14:24.320807 7083 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.178.1:36213 (error 108)
I20260806 21:14:24.321244 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 7398
I20260806 21:14:24.339488 7825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:24.339106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:36213 (local address 127.0.0.1:35836)
0806 21:14:24.339275 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:24.339387 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.1:36213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260806 21:14:24.339702 7083 catalog_manager.cc:4584] TS 5269f8bc3f5445f9af84a55eca845099 (127.16.178.1:36213): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 613ce14121fa4d4fb70f6e72a08f0814: Network error: Client connection negotiation failed: client connection to 127.16.178.1:36213: connect: Connection refused (error 111)
I20260806 21:14:24.344724 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 7622
I20260806 21:14:24.367481 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 7051
I20260806 21:14:24.384588 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:35423
--webserver_interface=127.16.178.62
--webserver_port=42461
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:46639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:35423
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:14:24.499404 7863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:14:24.499704 7863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:14:24.499745 7863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:14:24.503836 7863 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:14:24.503917 7863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:14:24.503937 7863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:14:24.503952 7863 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:14:24.503968 7863 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:14:24.509856 7863 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:46639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:35423
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:35423
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.178.62
--webserver_port=42461
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7863
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:14:24.511147 7863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:14:24.512432 7863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:14:24.521370 7869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:14:24.521370 7871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:14:24.521499 7863 server_base.cc:1034] running on GCE node
W20260806 21:14:24.521394 7873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:14:24.522392 7863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:14:24.523625 7863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:14:24.524808 7863 hybrid_clock.cc:648] HybridClock initialized: now 1786050864524790 us; error 36 us; skew 500 ppm
Aug 06 21:14:24 dist-test-slave-lgr5 krb5kdc[6191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050864, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:14:24.527962 7863 init.cc:377] Logged in from keytab as kudu/127.16.178.62@KRBTEST.COM (short username kudu)
I20260806 21:14:24.529072 7863 webserver.cc:460] Webserver started at http://127.16.178.62:42461/ using document root <none> and password file <none>
I20260806 21:14:24.529717 7863 fs_manager.cc:359] Metadata directory not provided
I20260806 21:14:24.529798 7863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:14:24.534117 7863 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 21:14:24.536442 7889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:24.537710 7863 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 21:14:24.537833 7863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "b5b9bac96afb4727a08a2d7f92db68ea"
format_stamp: "Formatted at 2026-08-06 21:14:22 on dist-test-slave-lgr5"
server_key: "0046ffbcfdd50ddfd0180b4c19f74b1b"
server_key_iv: "c9d1f2ed017a450a5c39e8ea9905dbff"
server_key_version: "encryptionkey@0"
I20260806 21:14:24.538425 7863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:14:24.556248 7863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:14:24.559423 7863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:14:24.559612 7863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:14:24.566800 7863 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.62:35423
I20260806 21:14:24.567130 7977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.62:35423 every 8 connection(s)
I20260806 21:14:24.567996 7863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:14:24.571802 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 7863
I20260806 21:14:24.572286 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.1:36213
--local_ip_for_outbound_sockets=127.16.178.1
--tserver_master_addrs=127.16.178.62:35423
--webserver_port=35361
--webserver_interface=127.16.178.1
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab
--principal=kudu/127.16.178.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:46639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 21:14:24.572636 7978 sys_catalog.cc:263] Verifying existing consensus state
I20260806 21:14:24.574419 7978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea: Bootstrap starting.
I20260806 21:14:24.599161 7978 log.cc:824] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea: Log is configured to *not* fsync() on all Append() calls
I20260806 21:14:24.614630 7978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 21:14:24.615150 7978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea: Bootstrap complete.
I20260806 21:14:24.620813 7978 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5b9bac96afb4727a08a2d7f92db68ea" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 35423 } }
I20260806 21:14:24.621371 7978 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5b9bac96afb4727a08a2d7f92db68ea, State: Initialized, Role: FOLLOWER
I20260806 21:14:24.621992 7978 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5b9bac96afb4727a08a2d7f92db68ea" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 35423 } }
I20260806 21:14:24.622157 7978 raft_consensus.cc:388] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:14:24.622231 7978 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:14:24.622344 7978 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [term 2 FOLLOWER]: Advancing to term 3
I20260806 21:14:24.625269 7978 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5b9bac96afb4727a08a2d7f92db68ea" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 35423 } }
I20260806 21:14:24.625703 7978 leader_election.cc:302] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b5b9bac96afb4727a08a2d7f92db68ea; no voters:
I20260806 21:14:24.626096 7978 leader_election.cc:288] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 21:14:24.626278 7983 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [term 3 FOLLOWER]: Leader election won for term 3
I20260806 21:14:24.626602 7983 raft_consensus.cc:686] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [term 3 LEADER]: Becoming Leader. State: Replica: b5b9bac96afb4727a08a2d7f92db68ea, State: Running, Role: LEADER
I20260806 21:14:24.626997 7983 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5b9bac96afb4727a08a2d7f92db68ea" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 35423 } }
I20260806 21:14:24.627487 7978 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:14:24.628539 7984 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "b5b9bac96afb4727a08a2d7f92db68ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5b9bac96afb4727a08a2d7f92db68ea" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 35423 } } }
I20260806 21:14:24.628674 7984 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [sys.catalog]: This master's current role is: LEADER
I20260806 21:14:24.629433 7985 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5b9bac96afb4727a08a2d7f92db68ea. Latest consensus state: current_term: 3 leader_uuid: "b5b9bac96afb4727a08a2d7f92db68ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5b9bac96afb4727a08a2d7f92db68ea" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 35423 } } }
I20260806 21:14:24.629778 7985 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea [sys.catalog]: This master's current role is: LEADER
I20260806 21:14:24.632887 8005 catalog_manager.cc:1259] Loaded cluster ID: e759654df3f34f68be7a171971ffed5f
I20260806 21:14:24.632944 8005 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea: loading cluster ID for follower catalog manager: success
W20260806 21:14:24.634099 8005 catalog_manager.cc:1576] T 00000000000000000000000000000000 P b5b9bac96afb4727a08a2d7f92db68ea: acquiring CA information for follower catalog manager: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
W20260806 21:14:24.634173 8005 catalog_manager.cc:873] Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO: failed to prepare follower catalog manager, will retry
I20260806 21:14:24.634276 7994 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:14:24.637856 7994 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:14:24.638366 7994 catalog_manager.cc:1259] Loaded cluster ID: e759654df3f34f68be7a171971ffed5f
I20260806 21:14:24.638425 7994 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260806 21:14:24.639111 7994 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fedfbc0e9ad google::LogMessage::Fail() at ??:0
@ 0x7fedfbc1188a google::LogMessage::SendToLog() at ??:0
@ 0x7fedfbc0e4de google::LogMessage::Flush() at ??:0
@ 0x7fedfbc0fa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fee0654dfdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fee0654ed2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fee0654b579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fee0658f640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fedfde603d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fedfde60c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fedfde623f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fedfde51512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fedff4956db start_thread at ??:0
@ 0x7fedfa70c61f clone at ??:0
W20260806 21:14:24.736748 7980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:14:24.737107 7980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:14:24.737157 7980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:14:24.741537 7980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:14:24.741616 7980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:14:24.741681 7980 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.1
I20260806 21:14:24.747306 7980 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:46639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab
--principal=kudu/127.16.178.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.1:36213
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.178.1
--webserver_port=35361
--tserver_master_addrs=127.16.178.62:35423
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7980
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.1
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:14:24.748543 7980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:14:24.749610 7980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:14:24.760229 8035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:14:24.760170 8032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:14:24.760361 7980 server_base.cc:1034] running on GCE node
W20260806 21:14:24.760361 8039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:14:24.761075 7980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:14:24.761750 7980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:14:24.762920 7980 hybrid_clock.cc:648] HybridClock initialized: now 1786050864762890 us; error 55 us; skew 500 ppm
Aug 06 21:14:24 dist-test-slave-lgr5 krb5kdc[6191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050864, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:14:24.765789 7980 init.cc:377] Logged in from keytab as kudu/127.16.178.1@KRBTEST.COM (short username kudu)
I20260806 21:14:24.766750 7980 webserver.cc:460] Webserver started at http://127.16.178.1:35361/ using document root <none> and password file <none>
I20260806 21:14:24.767314 7980 fs_manager.cc:359] Metadata directory not provided
I20260806 21:14:24.767376 7980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:14:24.771512 7980 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:14:24.773816 8050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:24.775242 7980 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 21:14:24.775345 7980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-0/wal
uuid: "5269f8bc3f5445f9af84a55eca845099"
format_stamp: "Formatted at 2026-08-06 21:14:22 on dist-test-slave-lgr5"
server_key: "7cd9a778070dd6249d86bd1408a5b3ba"
server_key_iv: "0066c2214e265ece377d8d83b9a17891"
server_key_version: "encryptionkey@0"
I20260806 21:14:24.775903 7980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:14:24.787715 7980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:14:24.791018 7980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:14:24.791208 7980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:14:24.791815 7980 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:14:24.793035 8060 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 21:14:24.799271 7980 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 21:14:24.799363 7980 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.000s sys 0.000s
I20260806 21:14:24.799433 7980 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 21:14:24.801268 7980 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 21:14:24.801324 7980 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260806 21:14:24.801692 8060 tablet_bootstrap.cc:492] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099: Bootstrap starting.
I20260806 21:14:24.811338 7980 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.1:36213
I20260806 21:14:24.811555 8219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.1:36213 every 8 connection(s)
I20260806 21:14:24.812520 7980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
I20260806 21:14:24.822693 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 7980
I20260806 21:14:24.823127 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.2:38433
--local_ip_for_outbound_sockets=127.16.178.2
--tserver_master_addrs=127.16.178.62:35423
--webserver_port=42233
--webserver_interface=127.16.178.2
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab
--principal=kudu/127.16.178.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:46639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 21:14:24.825013 8060 log.cc:824] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099: Log is configured to *not* fsync() on all Append() calls
I20260806 21:14:24.836205 8060 tablet_bootstrap.cc:492] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 21:14:24.836715 8060 tablet_bootstrap.cc:492] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099: Bootstrap complete.
I20260806 21:14:24.838068 8060 ts_tablet_manager.cc:1397] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099: Time spent bootstrapping tablet: real 0.037s user 0.026s sys 0.008s
I20260806 21:14:24.841728 8060 raft_consensus.cc:348] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2779b8373194b8ba7a735b2c5c20c15" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38433 } } peers { permanent_uuid: "07b6bdd51f164475a3477bad2ca26fed" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33729 } } peers { permanent_uuid: "5269f8bc3f5445f9af84a55eca845099" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36213 } }
I20260806 21:14:24.842397 8060 raft_consensus.cc:729] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5269f8bc3f5445f9af84a55eca845099, State: Initialized, Role: FOLLOWER
I20260806 21:14:24.843062 8060 consensus_queue.cc:260] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2779b8373194b8ba7a735b2c5c20c15" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38433 } } peers { permanent_uuid: "07b6bdd51f164475a3477bad2ca26fed" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33729 } } peers { permanent_uuid: "5269f8bc3f5445f9af84a55eca845099" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36213 } }
I20260806 21:14:24.844472 8060 ts_tablet_manager.cc:1428] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099: Time spent starting tablet: real 0.006s user 0.007s sys 0.001s
W20260806 21:14:24.869241 8223 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:14:24.814497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.1:54765)
0806 21:14:24.815002 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:24.815069 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0806 21:14:24.815896 (+ 827us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:14:24.868199 (+ 52303us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":127,"threads_started":1}
W20260806 21:14:24.869827 8221 heartbeater.cc:644] Failed to heartbeat to 127.16.178.62:35423 (0 consecutive failures): Network error: Failed to ping master at 127.16.178.62:35423: Client connection negotiation failed: client connection to 127.16.178.62:35423: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 21:14:24.870693 8223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:24.870367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.1:59111)
0806 21:14:24.870528 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:24.870617 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 21:14:24.871411 8223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:24.871193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.1:54841)
0806 21:14:24.871300 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:24.871352 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260806 21:14:24.871590 8221 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 21:14:24.942960 8224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:14:24.943300 8224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:14:24.943346 8224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:14:24.947501 8224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:14:24.947587 8224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:14:24.947649 8224 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.2
I20260806 21:14:24.953238 8224 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:46639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab
--principal=kudu/127.16.178.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.2:38433
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.178.2
--webserver_port=42233
--tserver_master_addrs=127.16.178.62:35423
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8224
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.2
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:14:24.954423 8224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:14:24.955524 8224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:14:24.966449 8244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:14:24.966447 8251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:14:24.966496 8224 server_base.cc:1034] running on GCE node
W20260806 21:14:24.966447 8246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:14:24.967172 8224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:14:24.967919 8224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:14:24.969089 8224 hybrid_clock.cc:648] HybridClock initialized: now 1786050864969073 us; error 60 us; skew 500 ppm
Aug 06 21:14:24 dist-test-slave-lgr5 krb5kdc[6191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050864, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:14:24.971915 8224 init.cc:377] Logged in from keytab as kudu/127.16.178.2@KRBTEST.COM (short username kudu)
I20260806 21:14:24.972926 8224 webserver.cc:460] Webserver started at http://127.16.178.2:42233/ using document root <none> and password file <none>
I20260806 21:14:24.973523 8224 fs_manager.cc:359] Metadata directory not provided
I20260806 21:14:24.973594 8224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:14:24.977826 8224 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.002s
I20260806 21:14:24.980306 8262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:24.981877 8224 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:14:24.982048 8224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-1/wal
uuid: "e2779b8373194b8ba7a735b2c5c20c15"
format_stamp: "Formatted at 2026-08-06 21:14:22 on dist-test-slave-lgr5"
server_key: "615f7c32a01ce888513a082d28ad21ff"
server_key_iv: "3804dc8dbd34e394e35749f1a656045e"
server_key_version: "encryptionkey@0"
I20260806 21:14:24.982695 8224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:14:25.013550 8224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:14:25.016842 8224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:14:25.017064 8224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:14:25.017710 8224 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:14:25.018580 8224 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:14:25.018628 8224 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:25.018682 8224 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:14:25.018712 8224 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:25.027842 8224 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.2:38433
I20260806 21:14:25.027887 8445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.2:38433 every 8 connection(s)
I20260806 21:14:25.028949 8224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
I20260806 21:14:25.033380 8448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:25.031276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.2:59425)
0806 21:14:25.031812 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:25.031969 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":130,"threads_started":1}
W20260806 21:14:25.033885 8446 heartbeater.cc:644] Failed to heartbeat to 127.16.178.62:35423 (0 consecutive failures): Network error: Failed to ping master at 127.16.178.62:35423: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
I20260806 21:14:25.034590 8448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:25.034320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.2:50707)
0806 21:14:25.034468 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:25.034524 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 21:14:25.035193 8448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:25.035006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.2:47681)
0806 21:14:25.035105 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:25.035146 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 21:14:25.035358 8446 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 21:14:25.035643 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 8224
I20260806 21:14:25.036046 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.3:33729
--local_ip_for_outbound_sockets=127.16.178.3
--tserver_master_addrs=127.16.178.62:35423
--webserver_port=45195
--webserver_interface=127.16.178.3
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab
--principal=kudu/127.16.178.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:46639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:14:25.162670 8449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:14:25.163014 8449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:14:25.163064 8449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:14:25.167289 8449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:14:25.167368 8449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:14:25.167433 8449 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.3
I20260806 21:14:25.173117 8449 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:46639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab
--principal=kudu/127.16.178.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.3:33729
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.178.3
--webserver_port=45195
--tserver_master_addrs=127.16.178.62:35423
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8449
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.3
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:14:25.174481 8449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:14:25.175629 8449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:14:25.186280 8456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:14:25.186276 8457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:14:25.186403 8461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:14:25.186283 8449 server_base.cc:1034] running on GCE node
I20260806 21:14:25.186996 8449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:14:25.187716 8449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:14:25.188903 8449 hybrid_clock.cc:648] HybridClock initialized: now 1786050865188871 us; error 42 us; skew 500 ppm
Aug 06 21:14:25 dist-test-slave-lgr5 krb5kdc[6191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050865, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:14:25.191778 8449 init.cc:377] Logged in from keytab as kudu/127.16.178.3@KRBTEST.COM (short username kudu)
I20260806 21:14:25.192821 8449 webserver.cc:460] Webserver started at http://127.16.178.3:45195/ using document root <none> and password file <none>
I20260806 21:14:25.193442 8449 fs_manager.cc:359] Metadata directory not provided
I20260806 21:14:25.193523 8449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:14:25.198009 8449 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260806 21:14:25.200384 8476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:25.201869 8449 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 21:14:25.202041 8449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-2/wal
uuid: "07b6bdd51f164475a3477bad2ca26fed"
format_stamp: "Formatted at 2026-08-06 21:14:22 on dist-test-slave-lgr5"
server_key: "a09fc69c01e6c42d66c3a02942325f84"
server_key_iv: "5da7c4883bf97482abb5b8fb88741fa9"
server_key_version: "encryptionkey@0"
I20260806 21:14:25.202709 8449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:14:25.225373 8449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:14:25.228952 8449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:14:25.229161 8449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:14:25.229802 8449 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:14:25.230711 8449 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:14:25.230762 8449 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:25.230819 8449 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:14:25.230847 8449 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:25.239686 8449 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.3:33729
I20260806 21:14:25.239706 8658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.3:33729 every 8 connection(s)
I20260806 21:14:25.240758 8449 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
I20260806 21:14:25.244753 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 8449
I20260806 21:14:25.244771 8663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:25.242904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.3:38929)
0806 21:14:25.243394 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:25.243522 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":118,"threads_started":1}
W20260806 21:14:25.245420 8660 heartbeater.cc:644] Failed to heartbeat to 127.16.178.62:35423 (0 consecutive failures): Network error: Failed to ping master at 127.16.178.62:35423: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
I20260806 21:14:25.246111 8663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:25.245848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.3:35479)
0806 21:14:25.246001 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:25.246053 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260806 21:14:25.246769 8663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:25.246518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.3:41757)
0806 21:14:25.246659 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:25.246712 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260806 21:14:25.246984 8660 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 21:14:25.872886 8666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:25.872388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.1:39483)
0806 21:14:25.872696 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:25.872762 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":86,"threads_started":1}
I20260806 21:14:26.036396 8667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:26.035937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.2:35487)
0806 21:14:26.036226 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:26.036282 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":77,"threads_started":1}
I20260806 21:14:26.247968 8669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:26.247514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.3:38905)
0806 21:14:26.247788 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:26.247849 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":86,"threads_started":1}
I20260806 21:14:26.492980 8670 raft_consensus.cc:482] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:14:26.493196 8670 raft_consensus.cc:504] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2779b8373194b8ba7a735b2c5c20c15" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38433 } } peers { permanent_uuid: "07b6bdd51f164475a3477bad2ca26fed" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33729 } } peers { permanent_uuid: "5269f8bc3f5445f9af84a55eca845099" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36213 } }
I20260806 21:14:26.494386 8670 leader_election.cc:288] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e2779b8373194b8ba7a735b2c5c20c15 (127.16.178.2:38433), 07b6bdd51f164475a3477bad2ca26fed (127.16.178.3:33729)
Aug 06 21:14:26 dist-test-slave-lgr5 krb5kdc[6191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050864, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for kudu/127.16.178.3@KRBTEST.COM
Aug 06 21:14:26 dist-test-slave-lgr5 krb5kdc[6191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050864, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for kudu/127.16.178.2@KRBTEST.COM
I20260806 21:14:26.507869 8673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:26.494826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:42073 (local address 127.16.178.2:38433)
0806 21:14:26.495345 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:26.495368 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:26.495409 (+ 41us) server_negotiation.cc:407] Connection header received
0806 21:14:26.496221 (+ 812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:26.496229 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:26.496589 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:26.496853 (+ 264us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:14:26.499246 (+ 2393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:26.500188 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:26.501002 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:26.501596 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:26.501715 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:26.503747 (+ 2032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:14:26.503752 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:14:26.503762 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:14:26.503789 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:14:26.505557 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:26.506019 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:26.506025 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:26.506030 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:26.506159 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:26.506501 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:26.506503 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:26.506504 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:26.506886 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:14:26.507012 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:26.507426 (+ 414us) server_negotiation.cc:299] Negotiation successful
0806 21:14:26.507678 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":85,"threads_started":1}
I20260806 21:14:26.507869 8675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:26.494683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.3:33729 (local address 127.16.178.1:37421)
0806 21:14:26.495422 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:26.495435 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:14:26.495535 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:14:26.497213 (+ 1678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:14:26.497216 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:14:26.497432 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:14:26.497950 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:26.497967 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:26.499199 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:26.499202 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:26.500015 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:26.500024 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:26.500810 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:26.500814 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:26.500986 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:26.501663 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:14:26.501685 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:14:26.503253 (+ 1568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:14:26.505297 (+ 2044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:26.505303 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:26.505318 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:26.505652 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:26.505985 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:26.505988 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:26.505990 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:26.506160 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:26.506882 (+ 722us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:14:26.506888 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:14:26.507134 (+ 246us) client_negotiation.cc:769] Sending connection context
0806 21:14:26.507296 (+ 162us) client_negotiation.cc:240] Negotiation successful
0806 21:14:26.507660 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678,"mutex_wait_us":451,"spinlock_wait_cycles":17024,"thread_start_us":66,"threads_started":1}
I20260806 21:14:26.507915 8674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:26.494867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:37421 (local address 127.16.178.3:33729)
0806 21:14:26.495224 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:26.495252 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:26.495521 (+ 269us) server_negotiation.cc:407] Connection header received
0806 21:14:26.496238 (+ 717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:26.496245 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:26.496589 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:26.496838 (+ 249us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:14:26.498118 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:26.499092 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:26.500142 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:26.500671 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:26.500847 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:26.503482 (+ 2635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:14:26.503490 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:14:26.503509 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:14:26.503545 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:14:26.505184 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:26.505753 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:26.505759 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:26.505765 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:26.505900 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:26.506279 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:26.506283 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:26.506284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:26.506758 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:14:26.506867 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:26.507472 (+ 605us) server_negotiation.cc:299] Negotiation successful
0806 21:14:26.507728 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":139,"threads_started":1}
I20260806 21:14:26.508669 8671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:26.494684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.2:38433 (local address 127.16.178.1:42073)
0806 21:14:26.495049 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:26.495064 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:14:26.495180 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:14:26.496946 (+ 1766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:14:26.496953 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:14:26.497437 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:14:26.497950 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:26.497966 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:26.500302 (+ 2336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:26.500305 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:26.500861 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:26.500872 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:26.501715 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:26.501719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:26.501801 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:26.502181 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:14:26.502196 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:14:26.503581 (+ 1385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:14:26.505663 (+ 2082us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:26.505665 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:26.505667 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:26.505901 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:26.506310 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:26.506313 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:26.506315 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:26.506400 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:26.506994 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:14:26.506996 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:14:26.507133 (+ 137us) client_negotiation.cc:769] Sending connection context
0806 21:14:26.507295 (+ 162us) client_negotiation.cc:240] Negotiation successful
0806 21:14:26.507732 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"mutex_wait_us":200,"spinlock_wait_cycles":16384,"thread_start_us":75,"threads_started":1}
W20260806 21:14:26.510357 8051 leader_election.cc:341] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 07b6bdd51f164475a3477bad2ca26fed (127.16.178.3:33729): Not found: Tablet not found: 613ce14121fa4d4fb70f6e72a08f0814
W20260806 21:14:26.510520 8054 leader_election.cc:341] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e2779b8373194b8ba7a735b2c5c20c15 (127.16.178.2:38433): Not found: Tablet not found: 613ce14121fa4d4fb70f6e72a08f0814
I20260806 21:14:26.510591 8054 leader_election.cc:302] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5269f8bc3f5445f9af84a55eca845099; no voters: 07b6bdd51f164475a3477bad2ca26fed, e2779b8373194b8ba7a735b2c5c20c15
I20260806 21:14:26.510816 8670 raft_consensus.cc:2726] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 21:14:26.874182 8671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:26.873850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.1:56087)
0806 21:14:26.874010 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:26.874086 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260806 21:14:27.037767 8676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:27.037254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.2:60129)
0806 21:14:27.037583 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:27.037655 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":87,"threads_started":1}
I20260806 21:14:27.249208 8677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:27.248772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.3:47527)
0806 21:14:27.249047 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:27.249109 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":95,"threads_started":1}
I20260806 21:14:27.875533 8678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:27.874965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.1:33733)
0806 21:14:27.875338 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:27.875406 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":129,"threads_started":1}
I20260806 21:14:28.039258 8679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:28.038751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.2:51765)
0806 21:14:28.039086 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:28.039148 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":97,"threads_started":1}
I20260806 21:14:28.195362 8680 raft_consensus.cc:482] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:14:28.195526 8680 raft_consensus.cc:504] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2779b8373194b8ba7a735b2c5c20c15" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38433 } } peers { permanent_uuid: "07b6bdd51f164475a3477bad2ca26fed" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33729 } } peers { permanent_uuid: "5269f8bc3f5445f9af84a55eca845099" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36213 } }
I20260806 21:14:28.195967 8680 leader_election.cc:288] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e2779b8373194b8ba7a735b2c5c20c15 (127.16.178.2:38433), 07b6bdd51f164475a3477bad2ca26fed (127.16.178.3:33729)
W20260806 21:14:28.196918 8051 leader_election.cc:341] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 07b6bdd51f164475a3477bad2ca26fed (127.16.178.3:33729): Not found: Tablet not found: 613ce14121fa4d4fb70f6e72a08f0814
W20260806 21:14:28.197073 8054 leader_election.cc:341] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e2779b8373194b8ba7a735b2c5c20c15 (127.16.178.2:38433): Not found: Tablet not found: 613ce14121fa4d4fb70f6e72a08f0814
I20260806 21:14:28.197170 8054 leader_election.cc:302] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5269f8bc3f5445f9af84a55eca845099; no voters: 07b6bdd51f164475a3477bad2ca26fed, e2779b8373194b8ba7a735b2c5c20c15
I20260806 21:14:28.197328 8680 raft_consensus.cc:2726] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 21:14:28.250494 8682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:28.250057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.3:39847)
0806 21:14:28.250336 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:28.250397 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":80,"threads_started":1}
I20260806 21:14:28.876936 8685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:28.876493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.1:38095)
0806 21:14:28.876767 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:28.876832 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":80,"threads_started":1}
I20260806 21:14:29.040603 8687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:29.040116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.2:42109)
0806 21:14:29.040423 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:29.040503 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":107,"threads_started":1}
I20260806 21:14:29.251715 8688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:29.251279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.3:59013)
0806 21:14:29.251562 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:29.251617 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":125,"threads_started":1}
I20260806 21:14:29.875545 8689 raft_consensus.cc:482] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:14:29.875696 8689 raft_consensus.cc:504] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2779b8373194b8ba7a735b2c5c20c15" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38433 } } peers { permanent_uuid: "07b6bdd51f164475a3477bad2ca26fed" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33729 } } peers { permanent_uuid: "5269f8bc3f5445f9af84a55eca845099" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36213 } }
I20260806 21:14:29.876206 8689 leader_election.cc:288] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e2779b8373194b8ba7a735b2c5c20c15 (127.16.178.2:38433), 07b6bdd51f164475a3477bad2ca26fed (127.16.178.3:33729)
W20260806 21:14:29.877067 8054 leader_election.cc:341] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e2779b8373194b8ba7a735b2c5c20c15 (127.16.178.2:38433): Not found: Tablet not found: 613ce14121fa4d4fb70f6e72a08f0814
W20260806 21:14:29.877188 8051 leader_election.cc:341] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 07b6bdd51f164475a3477bad2ca26fed (127.16.178.3:33729): Not found: Tablet not found: 613ce14121fa4d4fb70f6e72a08f0814
I20260806 21:14:29.877287 8051 leader_election.cc:302] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5269f8bc3f5445f9af84a55eca845099; no voters: 07b6bdd51f164475a3477bad2ca26fed, e2779b8373194b8ba7a735b2c5c20c15
I20260806 21:14:29.877439 8689 raft_consensus.cc:2726] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 21:14:29.878180 8690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:29.877747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.1:32821)
0806 21:14:29.878031 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:29.878095 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":65,"threads_started":1}
I20260806 21:14:30.041894 8691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:30.041399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.2:56157)
0806 21:14:30.041708 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:30.041783 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":76,"threads_started":1}
I20260806 21:14:30.252908 8692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:30.252454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.3:43715)
0806 21:14:30.252750 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:30.252810 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":77,"threads_started":1}
I20260806 21:14:30.879354 8693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:30.878902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.1:54389)
0806 21:14:30.879195 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:30.879253 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":114,"threads_started":1}
I20260806 21:14:31.043179 8697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:31.042682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.2:55017)
0806 21:14:31.042978 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:31.043061 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":88,"threads_started":1}
I20260806 21:14:31.254015 8698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:31.253614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.3:55507)
0806 21:14:31.253862 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:31.253916 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":75,"threads_started":1}
I20260806 21:14:31.880635 8700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:31.880193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.1:38897)
0806 21:14:31.880458 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:31.880527 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":86,"threads_started":1}
I20260806 21:14:32.044690 8701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:32.044149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.2:51489)
0806 21:14:32.044516 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:32.044583 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":93,"threads_started":1}
I20260806 21:14:32.255283 8702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:32.254762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.3:33827)
0806 21:14:32.255112 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:32.255179 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":91,"threads_started":1}
I20260806 21:14:32.881860 8703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:32.881411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.1:42637)
0806 21:14:32.881695 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:32.881758 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":92,"threads_started":1}
I20260806 21:14:33.045969 8707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:33.045519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.2:42275)
0806 21:14:33.045803 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:33.045864 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":79,"threads_started":1}
I20260806 21:14:33.256678 8708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:33.256175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.3:36635)
0806 21:14:33.256499 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:33.256570 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":92,"threads_started":1}
I20260806 21:14:33.883188 8709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:33.882724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.1:60993)
0806 21:14:33.883014 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:33.883082 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":94,"threads_started":1}
I20260806 21:14:34.047350 8710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:34.046848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.2:46147)
0806 21:14:34.047156 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:34.047234 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":87,"threads_started":1}
I20260806 21:14:34.258023 8711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:34.257547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.3:49927)
0806 21:14:34.257851 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:34.257918 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":83,"threads_started":1}
I20260806 21:14:34.269315 8712 raft_consensus.cc:482] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:14:34.269475 8712 raft_consensus.cc:504] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2779b8373194b8ba7a735b2c5c20c15" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38433 } } peers { permanent_uuid: "07b6bdd51f164475a3477bad2ca26fed" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33729 } } peers { permanent_uuid: "5269f8bc3f5445f9af84a55eca845099" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36213 } }
I20260806 21:14:34.269924 8712 leader_election.cc:288] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e2779b8373194b8ba7a735b2c5c20c15 (127.16.178.2:38433), 07b6bdd51f164475a3477bad2ca26fed (127.16.178.3:33729)
W20260806 21:14:34.270756 8051 leader_election.cc:341] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 07b6bdd51f164475a3477bad2ca26fed (127.16.178.3:33729): Not found: Tablet not found: 613ce14121fa4d4fb70f6e72a08f0814
W20260806 21:14:34.270886 8054 leader_election.cc:341] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e2779b8373194b8ba7a735b2c5c20c15 (127.16.178.2:38433): Not found: Tablet not found: 613ce14121fa4d4fb70f6e72a08f0814
I20260806 21:14:34.270995 8054 leader_election.cc:302] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5269f8bc3f5445f9af84a55eca845099; no voters: 07b6bdd51f164475a3477bad2ca26fed, e2779b8373194b8ba7a735b2c5c20c15
I20260806 21:14:34.271144 8712 raft_consensus.cc:2726] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 21:14:34.884594 8716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:34.884124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.1:48075)
0806 21:14:34.884419 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:34.884490 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":74,"threads_started":1}
I20260806 21:14:35.048882 8717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:35.048351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.2:60071)
0806 21:14:35.048691 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:35.048764 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":85,"threads_started":1}
I20260806 21:14:35.259332 8719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:35.258877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.3:45913)
0806 21:14:35.259154 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:35.259219 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":79,"threads_started":1}
I20260806 21:14:35.885948 8720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:35.885455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.1:43487)
0806 21:14:35.885785 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:35.885847 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":105,"threads_started":1}
I20260806 21:14:36.050606 8721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:36.050090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.2:46279)
0806 21:14:36.050410 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:36.050491 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":84,"threads_started":1}
I20260806 21:14:36.260740 8722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:36.260226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.3:36671)
0806 21:14:36.260563 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:36.260636 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":86,"threads_started":1}
I20260806 21:14:36.887192 8723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:36.886730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.1:40031)
0806 21:14:36.887010 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:36.887090 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":94,"threads_started":1}
I20260806 21:14:37.051877 8724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:37.051436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.2:35577)
0806 21:14:37.051715 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:37.051773 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":80,"threads_started":1}
I20260806 21:14:37.261981 8725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:37.261502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.3:49485)
0806 21:14:37.261812 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:37.261874 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":93,"threads_started":1}
I20260806 21:14:37.888556 8726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:37.888049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.1:44415)
0806 21:14:37.888390 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:37.888453 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":98,"threads_started":1}
I20260806 21:14:38.053206 8729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:38.052748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.2:52361)
0806 21:14:38.053042 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:38.053103 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":77,"threads_started":1}
I20260806 21:14:38.263371 8731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:38.262822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.3:46661)
0806 21:14:38.263176 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:38.263258 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":87,"threads_started":1}
I20260806 21:14:38.889843 8732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:38.889392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.1:48063)
0806 21:14:38.889675 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:38.889740 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":86,"threads_started":1}
I20260806 21:14:39.054548 8736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:39.054086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.2:37483)
0806 21:14:39.054380 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:39.054442 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":101,"threads_started":1}
I20260806 21:14:39.240922 8737 raft_consensus.cc:482] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:14:39.241074 8737 raft_consensus.cc:504] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2779b8373194b8ba7a735b2c5c20c15" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 38433 } } peers { permanent_uuid: "07b6bdd51f164475a3477bad2ca26fed" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33729 } } peers { permanent_uuid: "5269f8bc3f5445f9af84a55eca845099" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36213 } }
I20260806 21:14:39.241533 8737 leader_election.cc:288] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e2779b8373194b8ba7a735b2c5c20c15 (127.16.178.2:38433), 07b6bdd51f164475a3477bad2ca26fed (127.16.178.3:33729)
W20260806 21:14:39.242368 8051 leader_election.cc:341] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 07b6bdd51f164475a3477bad2ca26fed (127.16.178.3:33729): Not found: Tablet not found: 613ce14121fa4d4fb70f6e72a08f0814
W20260806 21:14:39.242494 8054 leader_election.cc:341] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e2779b8373194b8ba7a735b2c5c20c15 (127.16.178.2:38433): Not found: Tablet not found: 613ce14121fa4d4fb70f6e72a08f0814
I20260806 21:14:39.242558 8054 leader_election.cc:302] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5269f8bc3f5445f9af84a55eca845099; no voters: 07b6bdd51f164475a3477bad2ca26fed, e2779b8373194b8ba7a735b2c5c20c15
I20260806 21:14:39.242719 8737 raft_consensus.cc:2726] T 613ce14121fa4d4fb70f6e72a08f0814 P 5269f8bc3f5445f9af84a55eca845099 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 21:14:39.264819 8740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:39.264313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.3:54333)
0806 21:14:39.264650 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:39.264718 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":103,"threads_started":1}
I20260806 21:14:39.891362 8741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:39.890922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.1:33979)
0806 21:14:39.891198 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:39.891260 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":83,"threads_started":1}
I20260806 21:14:40.055913 8742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:40.055462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.2:40195)
0806 21:14:40.055755 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:40.055814 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":97,"threads_started":1}
I20260806 21:14:40.252265 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 7980
I20260806 21:14:40.267447 8743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:40.266074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35423 (local address 127.16.178.3:34471)
0806 21:14:40.267281 (+ 1207us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:40.267345 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:35423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1122,"thread_start_us":920,"threads_started":1}
I20260806 21:14:40.273334 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 8224
I20260806 21:14:40.300586 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 8449
2026-08-06T21:14:40Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19616 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:14:40 dist-test-slave-lgr5 krb5kdc[8747](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:14:40 dist-test-slave-lgr5 krb5kdc[8747](info): set up 2 sockets
Aug 06 21:14:40 dist-test-slave-lgr5 krb5kdc[8747](info): commencing operation
krb5kdc: starting...
W20260806 21:14:41.502775 17096 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.008s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Aug 06 21:14:41 dist-test-slave-lgr5 krb5kdc[8747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:14:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:14:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.178.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
I20260806 21:14:41.635830 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:37115
--webserver_interface=127.16.178.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:45599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:37115
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:14:41.750099 8766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:14:41.750361 8766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:14:41.750415 8766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:14:41.754384 8766 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 21:14:41.754441 8766 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 21:14:41.754460 8766 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 21:14:41.754478 8766 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:14:41.754503 8766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:14:41.754531 8766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:14:41.754560 8766 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:14:41.754588 8766 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:14:41.759781 8766 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:45599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:37115
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:37115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.178.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8766
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:14:41.760883 8766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:14:41.761901 8766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:14:41.770761 8781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:14:41.770844 8779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:14:41.770862 8766 server_base.cc:1034] running on GCE node
W20260806 21:14:41.770761 8776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:14:41.771682 8766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:14:41.772970 8766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:14:41.774165 8766 hybrid_clock.cc:648] HybridClock initialized: now 1786050881774145 us; error 36 us; skew 500 ppm
Aug 06 21:14:41 dist-test-slave-lgr5 krb5kdc[8747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050881, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:14:41.777063 8766 init.cc:377] Logged in from keytab as kudu/127.16.178.62@KRBTEST.COM (short username kudu)
I20260806 21:14:41.778024 8766 webserver.cc:460] Webserver started at http://127.16.178.62:44117/ using document root <none> and password file <none>
I20260806 21:14:41.778568 8766 fs_manager.cc:359] Metadata directory not provided
I20260806 21:14:41.778628 8766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:14:41.778806 8766 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:14:41.780922 8766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "82fdec997fa9475aa81fc138551e36e7"
format_stamp: "Formatted at 2026-08-06 21:14:41 on dist-test-slave-lgr5"
server_key: "e1354ebc86ec56fef3422989f77ade89"
server_key_iv: "437b13bb76db6a206a5f5e11e4f0132e"
server_key_version: "encryptionkey@0"
I20260806 21:14:41.781441 8766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "82fdec997fa9475aa81fc138551e36e7"
format_stamp: "Formatted at 2026-08-06 21:14:41 on dist-test-slave-lgr5"
server_key: "e1354ebc86ec56fef3422989f77ade89"
server_key_iv: "437b13bb76db6a206a5f5e11e4f0132e"
server_key_version: "encryptionkey@0"
I20260806 21:14:41.785385 8766 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:14:41.787899 8792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:41.789281 8766 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 21:14:41.789407 8766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "82fdec997fa9475aa81fc138551e36e7"
format_stamp: "Formatted at 2026-08-06 21:14:41 on dist-test-slave-lgr5"
server_key: "e1354ebc86ec56fef3422989f77ade89"
server_key_iv: "437b13bb76db6a206a5f5e11e4f0132e"
server_key_version: "encryptionkey@0"
I20260806 21:14:41.789506 8766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:14:41.805408 8766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:14:41.808878 8766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:14:41.809054 8766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:14:41.817106 8766 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.62:37115
I20260806 21:14:41.817495 8867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.62:37115 every 8 connection(s)
I20260806 21:14:41.818177 8766 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:14:41.821162 8868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:14:41.823272 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 8766
I20260806 21:14:41.823405 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:14:41.823719 17096 external_mini_cluster.cc:1428] Setting key cb1f6496acc67cd4d96803a3dd50f4a3
I20260806 21:14:41.827494 8868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7: Bootstrap starting.
I20260806 21:14:41.830461 8868 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7: Neither blocks nor log segments found. Creating new log.
I20260806 21:14:41.831338 8868 log.cc:824] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7: Log is configured to *not* fsync() on all Append() calls
Aug 06 21:14:41 dist-test-slave-lgr5 krb5kdc[8747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:14:41.833798 8868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7: No bootstrap required, opened a new log
I20260806 21:14:41.837900 8868 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fdec997fa9475aa81fc138551e36e7" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 37115 } }
I20260806 21:14:41.838171 8868 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:14:41.838240 8868 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82fdec997fa9475aa81fc138551e36e7, State: Initialized, Role: FOLLOWER
I20260806 21:14:41.838550 8871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:41.825259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51300 (local address 127.16.178.62:37115)
0806 21:14:41.825704 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:41.825721 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:41.825759 (+ 38us) server_negotiation.cc:407] Connection header received
0806 21:14:41.826489 (+ 730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:41.826508 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:41.826867 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:41.827164 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:14:41.828322 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:41.829375 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:41.830578 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:41.831107 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:41.831234 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:41.833869 (+ 2635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:14:41.833874 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:14:41.833890 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:14:41.833925 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:14:41.835565 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:41.835996 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:41.836002 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:41.836008 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:41.836164 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:41.836432 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:41.836435 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:41.836437 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:41.836867 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:14:41.837067 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:41.837367 (+ 300us) server_negotiation.cc:299] Negotiation successful
0806 21:14:41.837679 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":121,"threads_started":1}
I20260806 21:14:41.838737 8868 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fdec997fa9475aa81fc138551e36e7" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 37115 } }
I20260806 21:14:41.838881 8868 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:14:41.838948 8868 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:14:41.839046 8868 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:14:41.840258 8868 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fdec997fa9475aa81fc138551e36e7" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 37115 } }
I20260806 21:14:41.840636 8868 leader_election.cc:302] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82fdec997fa9475aa81fc138551e36e7; no voters:
I20260806 21:14:41.841022 8868 leader_election.cc:288] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:14:41.841105 8875 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:14:41.841329 8875 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [term 1 LEADER]: Becoming Leader. State: Replica: 82fdec997fa9475aa81fc138551e36e7, State: Running, Role: LEADER
I20260806 21:14:41.841694 8875 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fdec997fa9475aa81fc138551e36e7" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 37115 } }
I20260806 21:14:41.842208 8868 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:14:41.843390 8877 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82fdec997fa9475aa81fc138551e36e7. Latest consensus state: current_term: 1 leader_uuid: "82fdec997fa9475aa81fc138551e36e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fdec997fa9475aa81fc138551e36e7" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 37115 } } }
I20260806 21:14:41.843528 8877 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [sys.catalog]: This master's current role is: LEADER
I20260806 21:14:41.843847 8876 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "82fdec997fa9475aa81fc138551e36e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fdec997fa9475aa81fc138551e36e7" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 37115 } } }
I20260806 21:14:41.843997 8891 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:14:41.844033 8876 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [sys.catalog]: This master's current role is: LEADER
I20260806 21:14:41.847014 8891 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:14:41.853016 8891 catalog_manager.cc:1347] Generated new cluster ID: 0ac10d4413944f2293d8ec7cb6b3a3d3
I20260806 21:14:41.853089 8891 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:14:41.861704 8891 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:14:41.863082 8891 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:14:41.871374 8891 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7: Generated new TSK 0
I20260806 21:14:41.872229 8891 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.178.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab.
Entry for principal kudu/127.16.178.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab.
I20260806 21:14:41.903894 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.1:0
--local_ip_for_outbound_sockets=127.16.178.1
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:37115
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab
--principal=kudu/127.16.178.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:45599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:14:42.025866 8913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:14:42.026165 8913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:14:42.026211 8913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:14:42.030308 8913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:14:42.030386 8913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:14:42.030444 8913 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.1
I20260806 21:14:42.035941 8913 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:45599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab
--principal=kudu/127.16.178.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:37115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.1
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:14:42.037242 8913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:14:42.038407 8913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:14:42.048696 8932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:14:42.048696 8930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:14:42.048728 8926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:14:42.049053 8913 server_base.cc:1034] running on GCE node
I20260806 21:14:42.049530 8913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:14:42.050237 8913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:14:42.051434 8913 hybrid_clock.cc:648] HybridClock initialized: now 1786050882051410 us; error 31 us; skew 500 ppm
Aug 06 21:14:42 dist-test-slave-lgr5 krb5kdc[8747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050882, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:14:42.054940 8913 init.cc:377] Logged in from keytab as kudu/127.16.178.1@KRBTEST.COM (short username kudu)
I20260806 21:14:42.056257 8913 webserver.cc:460] Webserver started at http://127.16.178.1:40463/ using document root <none> and password file <none>
I20260806 21:14:42.056975 8913 fs_manager.cc:359] Metadata directory not provided
I20260806 21:14:42.057052 8913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:14:42.057268 8913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:14:42.059620 8913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-0/data/instance:
uuid: "f0dda022ce3140e08642813b191a3d19"
format_stamp: "Formatted at 2026-08-06 21:14:42 on dist-test-slave-lgr5"
server_key: "3ef4935787c7a2b933e6df7f4350afb1"
server_key_iv: "a8e0caa24230334a4dbd9d7a9fbfa49a"
server_key_version: "encryptionkey@0"
I20260806 21:14:42.060465 8913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance:
uuid: "f0dda022ce3140e08642813b191a3d19"
format_stamp: "Formatted at 2026-08-06 21:14:42 on dist-test-slave-lgr5"
server_key: "3ef4935787c7a2b933e6df7f4350afb1"
server_key_iv: "a8e0caa24230334a4dbd9d7a9fbfa49a"
server_key_version: "encryptionkey@0"
I20260806 21:14:42.066329 8913 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 21:14:42.070516 8940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:42.072474 8913 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 21:14:42.072726 8913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-0/wal
uuid: "f0dda022ce3140e08642813b191a3d19"
format_stamp: "Formatted at 2026-08-06 21:14:42 on dist-test-slave-lgr5"
server_key: "3ef4935787c7a2b933e6df7f4350afb1"
server_key_iv: "a8e0caa24230334a4dbd9d7a9fbfa49a"
server_key_version: "encryptionkey@0"
I20260806 21:14:42.072988 8913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:14:42.096487 8913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:14:42.099849 8913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:14:42.100039 8913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:14:42.100719 8913 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:14:42.101600 8913 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:14:42.101649 8913 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:42.101703 8913 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:14:42.101730 8913 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:42.110455 8913 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.1:36913
I20260806 21:14:42.110479 9114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.1:36913 every 8 connection(s)
I20260806 21:14:42.111367 8913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
I20260806 21:14:42.113620 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 8913
I20260806 21:14:42.113757 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance
I20260806 21:14:42.114107 17096 external_mini_cluster.cc:1428] Setting key 14deb97daded889319ccf555697a859b
Aug 06 21:14:42 dist-test-slave-lgr5 krb5kdc[8747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050882, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.178.2@KRBTEST.COM; defaulting to no policy
I20260806 21:14:42.126006 8871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:42.113239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:36541 (local address 127.16.178.62:37115)
0806 21:14:42.113398 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:42.113402 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:42.114393 (+ 991us) server_negotiation.cc:407] Connection header received
0806 21:14:42.116033 (+ 1640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:42.116036 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:42.116123 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:42.116186 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:14:42.117537 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:42.118086 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:42.119248 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:42.119481 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:42.119583 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:42.122163 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:14:42.122166 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:14:42.122168 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:14:42.122194 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:14:42.123571 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:42.124237 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:42.124240 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:42.124242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:42.124340 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:42.124708 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:42.124712 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:42.124713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:42.124969 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:14:42.125070 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:42.125696 (+ 626us) server_negotiation.cc:299] Negotiation successful
0806 21:14:42.125848 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 21:14:42.126937 9119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:42.113672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.1:36541)
0806 21:14:42.114249 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:42.114294 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 21:14:42.115824 (+ 1530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:14:42.116452 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:14:42.116460 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:14:42.116857 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:14:42.117378 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:42.117393 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:42.118211 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:42.118214 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:42.119126 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:42.119136 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:42.119644 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:42.119648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:42.119818 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:42.120428 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:14:42.120445 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:14:42.121982 (+ 1537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:14:42.123694 (+ 1712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:42.123701 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:42.123721 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:42.124000 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:42.124441 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:42.124444 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:42.124446 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:42.124611 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:42.125093 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:14:42.125100 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:14:42.125408 (+ 308us) client_negotiation.cc:769] Sending connection context
0806 21:14:42.125611 (+ 203us) client_negotiation.cc:240] Negotiation successful
0806 21:14:42.125940 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":155,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.2@KRBTEST.COM" created.
I20260806 21:14:42.132190 9115 heartbeater.cc:347] Connected to a master server at 127.16.178.62:37115
I20260806 21:14:42.132619 9115 heartbeater.cc:464] Registering TS with master...
I20260806 21:14:42.133445 9115 heartbeater.cc:511] Master 127.16.178.62:37115 requested a full tablet report, sending...
I20260806 21:14:42.136125 8810 ts_manager.cc:194] Registered new tserver with Master: f0dda022ce3140e08642813b191a3d19 (127.16.178.1:36913)
I20260806 21:14:42.138056 8810 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.1@KRBTEST.COM'} at 127.16.178.1:36541
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab.
Entry for principal kudu/127.16.178.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab.
I20260806 21:14:42.148707 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.2:0
--local_ip_for_outbound_sockets=127.16.178.2
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:37115
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab
--principal=kudu/127.16.178.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:45599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:14:42.271853 9123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:14:42.272162 9123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:14:42.272207 9123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:14:42.276345 9123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:14:42.276412 9123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:14:42.276482 9123 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.2
I20260806 21:14:42.282063 9123 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:45599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab
--principal=kudu/127.16.178.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:37115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.2
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:14:42.283213 9123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:14:42.284372 9123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:14:42.294844 9134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:14:42.294903 9136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:14:42.294837 9133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:14:42.294894 9123 server_base.cc:1034] running on GCE node
I20260806 21:14:42.295635 9123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:14:42.296432 9123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:14:42.297612 9123 hybrid_clock.cc:648] HybridClock initialized: now 1786050882297586 us; error 46 us; skew 500 ppm
Aug 06 21:14:42 dist-test-slave-lgr5 krb5kdc[8747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050882, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:14:42.300688 9123 init.cc:377] Logged in from keytab as kudu/127.16.178.2@KRBTEST.COM (short username kudu)
I20260806 21:14:42.301730 9123 webserver.cc:460] Webserver started at http://127.16.178.2:44737/ using document root <none> and password file <none>
I20260806 21:14:42.302347 9123 fs_manager.cc:359] Metadata directory not provided
I20260806 21:14:42.302433 9123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:14:42.302659 9123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:14:42.305061 9123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-1/data/instance:
uuid: "1522aa3f9fe54b1d95e66ff625693695"
format_stamp: "Formatted at 2026-08-06 21:14:42 on dist-test-slave-lgr5"
server_key: "3733ff686be7eb1cfb4140b162b931f4"
server_key_iv: "84e3ed641f1ef6253e1969d8d22d01b0"
server_key_version: "encryptionkey@0"
I20260806 21:14:42.305696 9123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance:
uuid: "1522aa3f9fe54b1d95e66ff625693695"
format_stamp: "Formatted at 2026-08-06 21:14:42 on dist-test-slave-lgr5"
server_key: "3733ff686be7eb1cfb4140b162b931f4"
server_key_iv: "84e3ed641f1ef6253e1969d8d22d01b0"
server_key_version: "encryptionkey@0"
I20260806 21:14:42.310221 9123 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:14:42.313077 9146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:42.314417 9123 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 21:14:42.314563 9123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-1/wal
uuid: "1522aa3f9fe54b1d95e66ff625693695"
format_stamp: "Formatted at 2026-08-06 21:14:42 on dist-test-slave-lgr5"
server_key: "3733ff686be7eb1cfb4140b162b931f4"
server_key_iv: "84e3ed641f1ef6253e1969d8d22d01b0"
server_key_version: "encryptionkey@0"
I20260806 21:14:42.314687 9123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:14:42.329608 9123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:14:42.332937 9123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:14:42.333133 9123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:14:42.333776 9123 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:14:42.334666 9123 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:14:42.334717 9123 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:42.334774 9123 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:14:42.334800 9123 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:42.344141 9123 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.2:37533
I20260806 21:14:42.344254 9326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.2:37533 every 8 connection(s)
I20260806 21:14:42.345157 9123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
I20260806 21:14:42.347327 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 9123
I20260806 21:14:42.347460 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance
I20260806 21:14:42.347757 17096 external_mini_cluster.cc:1428] Setting key 1d19d54241cdc136d16b6a9b48931bde
Aug 06 21:14:42 dist-test-slave-lgr5 krb5kdc[8747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050882, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.2@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.178.3@KRBTEST.COM; defaulting to no policy
I20260806 21:14:42.359201 8871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:42.346904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:33219 (local address 127.16.178.62:37115)
0806 21:14:42.347022 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:42.347026 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:42.347908 (+ 882us) server_negotiation.cc:407] Connection header received
0806 21:14:42.349460 (+ 1552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:42.349463 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:42.349516 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:42.349599 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:14:42.350731 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:42.351466 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:42.353043 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:42.353259 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:42.353351 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:42.355949 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:14:42.355953 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:14:42.355955 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:14:42.355980 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:14:42.357258 (+ 1278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:42.357740 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:42.357743 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:42.357745 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:42.357819 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:42.358136 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:42.358139 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:42.358141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:42.358320 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:14:42.358414 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:42.358964 (+ 550us) server_negotiation.cc:299] Negotiation successful
0806 21:14:42.359072 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 21:14:42.360023 9329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:42.347241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.2:33219)
0806 21:14:42.347737 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:42.347792 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 21:14:42.349254 (+ 1462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:14:42.349728 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:14:42.349734 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:14:42.350101 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:14:42.350583 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:42.350597 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:42.351593 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:42.351597 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:42.352947 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:42.352957 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:42.353418 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:42.353422 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:42.353589 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:42.354161 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:14:42.354177 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:14:42.355776 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:14:42.357368 (+ 1592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:42.357374 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:42.357397 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:42.357644 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:42.357905 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:42.357908 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:42.357910 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:42.358047 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:42.358445 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:14:42.358453 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:14:42.358747 (+ 294us) client_negotiation.cc:769] Sending connection context
0806 21:14:42.358917 (+ 170us) client_negotiation.cc:240] Negotiation successful
0806 21:14:42.359172 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":162,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.3@KRBTEST.COM" created.
I20260806 21:14:42.361261 9327 heartbeater.cc:347] Connected to a master server at 127.16.178.62:37115
I20260806 21:14:42.361546 9327 heartbeater.cc:464] Registering TS with master...
I20260806 21:14:42.362073 9327 heartbeater.cc:511] Master 127.16.178.62:37115 requested a full tablet report, sending...
I20260806 21:14:42.363189 8810 ts_manager.cc:194] Registered new tserver with Master: 1522aa3f9fe54b1d95e66ff625693695 (127.16.178.2:37533)
I20260806 21:14:42.363793 8810 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.2@KRBTEST.COM'} at 127.16.178.2:33219
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab.
Entry for principal kudu/127.16.178.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab.
I20260806 21:14:42.379460 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.3:0
--local_ip_for_outbound_sockets=127.16.178.3
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:37115
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab
--principal=kudu/127.16.178.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:45599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:14:42.497284 9334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:14:42.497579 9334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:14:42.497622 9334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:14:42.501664 9334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:14:42.501734 9334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:14:42.501796 9334 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.3
I20260806 21:14:42.507347 9334 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:45599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab
--principal=kudu/127.16.178.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:37115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9334
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.3
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:14:42.508508 9334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:14:42.509536 9334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:14:42.519660 9346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:14:42.519660 9342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:14:42.519660 9340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:14:42.519927 9334 server_base.cc:1034] running on GCE node
I20260806 21:14:42.520459 9334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:14:42.521109 9334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:14:42.522287 9334 hybrid_clock.cc:648] HybridClock initialized: now 1786050882522269 us; error 30 us; skew 500 ppm
Aug 06 21:14:42 dist-test-slave-lgr5 krb5kdc[8747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050882, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:14:42.525043 9334 init.cc:377] Logged in from keytab as kudu/127.16.178.3@KRBTEST.COM (short username kudu)
I20260806 21:14:42.525928 9334 webserver.cc:460] Webserver started at http://127.16.178.3:44075/ using document root <none> and password file <none>
I20260806 21:14:42.526481 9334 fs_manager.cc:359] Metadata directory not provided
I20260806 21:14:42.526549 9334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:14:42.526742 9334 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:14:42.528877 9334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-2/data/instance:
uuid: "6da22627b28b4b19bef57184d3167576"
format_stamp: "Formatted at 2026-08-06 21:14:42 on dist-test-slave-lgr5"
server_key: "35923fcda49e926fa5e11018e8e2146a"
server_key_iv: "6b2e9dc722f6d970b1a5caa219436a04"
server_key_version: "encryptionkey@0"
I20260806 21:14:42.529417 9334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance:
uuid: "6da22627b28b4b19bef57184d3167576"
format_stamp: "Formatted at 2026-08-06 21:14:42 on dist-test-slave-lgr5"
server_key: "35923fcda49e926fa5e11018e8e2146a"
server_key_iv: "6b2e9dc722f6d970b1a5caa219436a04"
server_key_version: "encryptionkey@0"
I20260806 21:14:42.533135 9334 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 21:14:42.535705 9354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:42.537051 9334 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:14:42.537175 9334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-2/wal
uuid: "6da22627b28b4b19bef57184d3167576"
format_stamp: "Formatted at 2026-08-06 21:14:42 on dist-test-slave-lgr5"
server_key: "35923fcda49e926fa5e11018e8e2146a"
server_key_iv: "6b2e9dc722f6d970b1a5caa219436a04"
server_key_version: "encryptionkey@0"
I20260806 21:14:42.537297 9334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:14:42.549639 9334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:14:42.552639 9334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:14:42.552835 9334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:14:42.553465 9334 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:14:42.554365 9334 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:14:42.554423 9334 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:42.554481 9334 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:14:42.554505 9334 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:14:42.563437 9334 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.3:46401
I20260806 21:14:42.563511 9525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.3:46401 every 8 connection(s)
I20260806 21:14:42.564445 9334 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
I20260806 21:14:42.566840 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 9334
I20260806 21:14:42.566972 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance
I20260806 21:14:42.567256 17096 external_mini_cluster.cc:1428] Setting key 1fb815e78eb4b8458fcb3a32c2c83e40
Aug 06 21:14:42 dist-test-slave-lgr5 krb5kdc[8747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050882, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.3@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:14:42.578715 8871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:42.566383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:37515 (local address 127.16.178.62:37115)
0806 21:14:42.566575 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:42.566580 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:42.567561 (+ 981us) server_negotiation.cc:407] Connection header received
0806 21:14:42.569255 (+ 1694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:42.569259 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:42.569319 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:42.569392 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:14:42.570784 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:42.571328 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:42.572680 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:42.572907 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:42.573007 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:42.575445 (+ 2438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:14:42.575448 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:14:42.575450 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:14:42.575477 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:14:42.576655 (+ 1178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:42.577145 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:42.577148 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:42.577150 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:42.577227 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:42.577572 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:42.577575 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:42.577576 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:42.577806 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:14:42.577893 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:42.578410 (+ 517us) server_negotiation.cc:299] Negotiation successful
0806 21:14:42.578542 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 21:14:42.579640 9529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:42.566821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.3:37515)
0806 21:14:42.567365 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:42.567436 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0806 21:14:42.569018 (+ 1582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:14:42.569611 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:14:42.569622 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:14:42.570037 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:14:42.570630 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:42.570647 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:42.571471 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:42.571475 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:42.572560 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:42.572572 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:42.573071 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:42.573075 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:42.573249 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:42.573810 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:14:42.573828 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:14:42.575256 (+ 1428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:14:42.576766 (+ 1510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:42.576773 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:42.576790 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:42.577048 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:42.577311 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:14:42.577314 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:14:42.577316 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:14:42.577465 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:14:42.577906 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:14:42.577913 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:14:42.578173 (+ 260us) client_negotiation.cc:769] Sending connection context
0806 21:14:42.578349 (+ 176us) client_negotiation.cc:240] Negotiation successful
0806 21:14:42.578699 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":156,"threads_started":1}
I20260806 21:14:42.580930 9526 heartbeater.cc:347] Connected to a master server at 127.16.178.62:37115
I20260806 21:14:42.581185 9526 heartbeater.cc:464] Registering TS with master...
I20260806 21:14:42.581696 9526 heartbeater.cc:511] Master 127.16.178.62:37115 requested a full tablet report, sending...
I20260806 21:14:42.582711 8819 ts_manager.cc:194] Registered new tserver with Master: 6da22627b28b4b19bef57184d3167576 (127.16.178.3:46401)
I20260806 21:14:42.583274 8819 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.3@KRBTEST.COM'} at 127.16.178.3:37515
I20260806 21:14:42.593830 17096 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 21:14:42.602650 8871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:42.596025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51316 (local address 127.16.178.62:37115)
0806 21:14:42.596202 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:42.596218 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:42.596369 (+ 151us) server_negotiation.cc:407] Connection header received
0806 21:14:42.596487 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:42.596489 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:42.596538 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:42.596608 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:14:42.597256 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:42.597768 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:42.598533 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:42.598736 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:42.598803 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:42.600131 (+ 1328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:14:42.600134 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:14:42.600137 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:14:42.600156 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:14:42.601283 (+ 1127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:42.601649 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:42.601651 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:42.601653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:42.601723 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:14:42.601912 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:14:42.601914 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:14:42.601915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:14:42.602141 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:14:42.602241 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:42.602399 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 21:14:42.602494 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 21:14:42.605839 8819 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51316:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:14:42.609055 8819 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:14:42.624254 9544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:42.618229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46306 (local address 127.16.178.2:37533)
0806 21:14:42.618685 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:42.618697 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:42.619211 (+ 514us) server_negotiation.cc:407] Connection header received
0806 21:14:42.619515 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:42.619521 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:42.619770 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:42.619896 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:14:42.620581 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:42.621357 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:42.622907 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:42.623532 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:42.623633 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:42.623794 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:42.623959 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 21:14:42.624060 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":131,"threads_started":1}
I20260806 21:14:42.624254 9541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:42.618082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.2:37533 (local address 127.0.0.1:46306)
0806 21:14:42.618986 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:42.619120 (+ 134us) client_negotiation.cc:174] Beginning negotiation
0806 21:14:42.619331 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:14:42.620141 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:14:42.620148 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:14:42.620185 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:14:42.620454 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:42.620464 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:42.621493 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:42.621497 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:42.622762 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:42.622774 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:42.623650 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:42.623654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:42.623768 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:42.623808 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 21:14:42.623918 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 21:14:42.624060 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":728,"mutex_wait_us":172,"spinlock_wait_cycles":7936,"thread_start_us":95,"threads_started":1}
I20260806 21:14:42.624449 9545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:42.618198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.3:46401 (local address 127.0.0.1:45090)
0806 21:14:42.619323 (+ 1125us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:42.619334 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 21:14:42.619409 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:14:42.620141 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:14:42.620148 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:14:42.620186 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:14:42.620478 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:42.620485 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:42.621963 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:42.621967 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:42.622939 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:42.622950 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:42.623989 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:42.623992 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:42.624107 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:42.624134 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 21:14:42.624185 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 21:14:42.624255 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1062,"mutex_wait_us":120,"spinlock_wait_cycles":6912,"thread_start_us":131,"threads_started":1}
I20260806 21:14:42.625073 9547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:42.618925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45090 (local address 127.16.178.3:46401)
0806 21:14:42.619673 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:42.619683 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:42.619705 (+ 22us) server_negotiation.cc:407] Connection header received
0806 21:14:42.619769 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:42.619773 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:42.619950 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:42.620125 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:14:42.620609 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:42.621807 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:42.623065 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:42.623854 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:42.624631 (+ 777us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:42.624774 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:42.624826 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 21:14:42.624916 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":648,"thread_start_us":110,"threads_started":1}
I20260806 21:14:42.625379 9540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:42.618430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:36913 (local address 127.0.0.1:51656)
0806 21:14:42.619610 (+ 1180us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:42.619621 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 21:14:42.619759 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:14:42.620307 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:14:42.620310 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:14:42.620328 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:14:42.620503 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:42.620510 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:42.621963 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:42.621967 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:42.623746 (+ 1779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:42.623758 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:42.624999 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:42.625003 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:42.625097 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:42.625121 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 21:14:42.625193 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 21:14:42.625259 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1122,"mutex_wait_us":7,"thread_start_us":95,"threads_started":1}
I20260806 21:14:42.625806 9548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:42.618928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51656 (local address 127.16.178.1:36913)
0806 21:14:42.619472 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:42.619488 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:42.619696 (+ 208us) server_negotiation.cc:407] Connection header received
0806 21:14:42.619911 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:42.619916 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:42.620133 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:42.620265 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:14:42.620664 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:42.621805 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:42.623896 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:42.624710 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:42.624811 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:42.624933 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:42.625556 (+ 623us) server_negotiation.cc:299] Negotiation successful
0806 21:14:42.625656 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":127,"threads_started":1}
I20260806 21:14:42.628209 9223 tablet_service.cc:1467] Processing CreateTablet for tablet 93ab0a5ed55649caa16eaf0cffbf4465 (DEFAULT_TABLE table=test-table [id=2d56910231104bd5a074b5a8dabdbd35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:14:42.628381 9427 tablet_service.cc:1467] Processing CreateTablet for tablet 93ab0a5ed55649caa16eaf0cffbf4465 (DEFAULT_TABLE table=test-table [id=2d56910231104bd5a074b5a8dabdbd35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:14:42.628727 9013 tablet_service.cc:1467] Processing CreateTablet for tablet 93ab0a5ed55649caa16eaf0cffbf4465 (DEFAULT_TABLE table=test-table [id=2d56910231104bd5a074b5a8dabdbd35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:14:42.629575 9427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93ab0a5ed55649caa16eaf0cffbf4465. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:14:42.629586 9223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93ab0a5ed55649caa16eaf0cffbf4465. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:14:42.629671 9013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93ab0a5ed55649caa16eaf0cffbf4465. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:14:42.636466 9549 tablet_bootstrap.cc:492] T 93ab0a5ed55649caa16eaf0cffbf4465 P 1522aa3f9fe54b1d95e66ff625693695: Bootstrap starting.
I20260806 21:14:42.637867 9550 tablet_bootstrap.cc:492] T 93ab0a5ed55649caa16eaf0cffbf4465 P f0dda022ce3140e08642813b191a3d19: Bootstrap starting.
I20260806 21:14:42.639150 9549 tablet_bootstrap.cc:654] T 93ab0a5ed55649caa16eaf0cffbf4465 P 1522aa3f9fe54b1d95e66ff625693695: Neither blocks nor log segments found. Creating new log.
I20260806 21:14:42.639583 9550 tablet_bootstrap.cc:654] T 93ab0a5ed55649caa16eaf0cffbf4465 P f0dda022ce3140e08642813b191a3d19: Neither blocks nor log segments found. Creating new log.
I20260806 21:14:42.639912 9549 log.cc:824] T 93ab0a5ed55649caa16eaf0cffbf4465 P 1522aa3f9fe54b1d95e66ff625693695: Log is configured to *not* fsync() on all Append() calls
I20260806 21:14:42.640362 9550 log.cc:824] T 93ab0a5ed55649caa16eaf0cffbf4465 P f0dda022ce3140e08642813b191a3d19: Log is configured to *not* fsync() on all Append() calls
I20260806 21:14:42.640676 9551 tablet_bootstrap.cc:492] T 93ab0a5ed55649caa16eaf0cffbf4465 P 6da22627b28b4b19bef57184d3167576: Bootstrap starting.
I20260806 21:14:42.642446 9550 tablet_bootstrap.cc:492] T 93ab0a5ed55649caa16eaf0cffbf4465 P f0dda022ce3140e08642813b191a3d19: No bootstrap required, opened a new log
I20260806 21:14:42.642465 9551 tablet_bootstrap.cc:654] T 93ab0a5ed55649caa16eaf0cffbf4465 P 6da22627b28b4b19bef57184d3167576: Neither blocks nor log segments found. Creating new log.
I20260806 21:14:42.642628 9550 ts_tablet_manager.cc:1397] T 93ab0a5ed55649caa16eaf0cffbf4465 P f0dda022ce3140e08642813b191a3d19: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 21:14:42.643524 9551 log.cc:824] T 93ab0a5ed55649caa16eaf0cffbf4465 P 6da22627b28b4b19bef57184d3167576: Log is configured to *not* fsync() on all Append() calls
I20260806 21:14:42.645248 9549 tablet_bootstrap.cc:492] T 93ab0a5ed55649caa16eaf0cffbf4465 P 1522aa3f9fe54b1d95e66ff625693695: No bootstrap required, opened a new log
I20260806 21:14:42.645565 9551 tablet_bootstrap.cc:492] T 93ab0a5ed55649caa16eaf0cffbf4465 P 6da22627b28b4b19bef57184d3167576: No bootstrap required, opened a new log
I20260806 21:14:42.645711 9549 ts_tablet_manager.cc:1397] T 93ab0a5ed55649caa16eaf0cffbf4465 P 1522aa3f9fe54b1d95e66ff625693695: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260806 21:14:42.645766 9551 ts_tablet_manager.cc:1397] T 93ab0a5ed55649caa16eaf0cffbf4465 P 6da22627b28b4b19bef57184d3167576: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 21:14:42.646394 9550 raft_consensus.cc:348] T 93ab0a5ed55649caa16eaf0cffbf4465 P f0dda022ce3140e08642813b191a3d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1522aa3f9fe54b1d95e66ff625693695" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 37533 } } peers { permanent_uuid: "6da22627b28b4b19bef57184d3167576" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 46401 } } peers { permanent_uuid: "f0dda022ce3140e08642813b191a3d19" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36913 } }
I20260806 21:14:42.646634 9550 raft_consensus.cc:374] T 93ab0a5ed55649caa16eaf0cffbf4465 P f0dda022ce3140e08642813b191a3d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:14:42.646718 9550 raft_consensus.cc:729] T 93ab0a5ed55649caa16eaf0cffbf4465 P f0dda022ce3140e08642813b191a3d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0dda022ce3140e08642813b191a3d19, State: Initialized, Role: FOLLOWER
I20260806 21:14:42.647171 9550 consensus_queue.cc:260] T 93ab0a5ed55649caa16eaf0cffbf4465 P f0dda022ce3140e08642813b191a3d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1522aa3f9fe54b1d95e66ff625693695" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 37533 } } peers { permanent_uuid: "6da22627b28b4b19bef57184d3167576" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 46401 } } peers { permanent_uuid: "f0dda022ce3140e08642813b191a3d19" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36913 } }
I20260806 21:14:42.648291 9550 ts_tablet_manager.cc:1428] T 93ab0a5ed55649caa16eaf0cffbf4465 P f0dda022ce3140e08642813b191a3d19: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 21:14:42.648734 9115 heartbeater.cc:503] Master 127.16.178.62:37115 was elected leader, sending a full tablet report...
I20260806 21:14:42.649708 9549 raft_consensus.cc:348] T 93ab0a5ed55649caa16eaf0cffbf4465 P 1522aa3f9fe54b1d95e66ff625693695 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1522aa3f9fe54b1d95e66ff625693695" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 37533 } } peers { permanent_uuid: "6da22627b28b4b19bef57184d3167576" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 46401 } } peers { permanent_uuid: "f0dda022ce3140e08642813b191a3d19" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36913 } }
I20260806 21:14:42.649721 9551 raft_consensus.cc:348] T 93ab0a5ed55649caa16eaf0cffbf4465 P 6da22627b28b4b19bef57184d3167576 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1522aa3f9fe54b1d95e66ff625693695" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 37533 } } peers { permanent_uuid: "6da22627b28b4b19bef57184d3167576" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 46401 } } peers { permanent_uuid: "f0dda022ce3140e08642813b191a3d19" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36913 } }
I20260806 21:14:42.650018 9551 raft_consensus.cc:374] T 93ab0a5ed55649caa16eaf0cffbf4465 P 6da22627b28b4b19bef57184d3167576 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:14:42.650018 9549 raft_consensus.cc:374] T 93ab0a5ed55649caa16eaf0cffbf4465 P 1522aa3f9fe54b1d95e66ff625693695 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:14:42.650089 9549 raft_consensus.cc:729] T 93ab0a5ed55649caa16eaf0cffbf4465 P 1522aa3f9fe54b1d95e66ff625693695 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1522aa3f9fe54b1d95e66ff625693695, State: Initialized, Role: FOLLOWER
I20260806 21:14:42.650117 9551 raft_consensus.cc:729] T 93ab0a5ed55649caa16eaf0cffbf4465 P 6da22627b28b4b19bef57184d3167576 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6da22627b28b4b19bef57184d3167576, State: Initialized, Role: FOLLOWER
I20260806 21:14:42.650586 9551 consensus_queue.cc:260] T 93ab0a5ed55649caa16eaf0cffbf4465 P 6da22627b28b4b19bef57184d3167576 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1522aa3f9fe54b1d95e66ff625693695" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 37533 } } peers { permanent_uuid: "6da22627b28b4b19bef57184d3167576" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 46401 } } peers { permanent_uuid: "f0dda022ce3140e08642813b191a3d19" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36913 } }
I20260806 21:14:42.650586 9549 consensus_queue.cc:260] T 93ab0a5ed55649caa16eaf0cffbf4465 P 1522aa3f9fe54b1d95e66ff625693695 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1522aa3f9fe54b1d95e66ff625693695" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 37533 } } peers { permanent_uuid: "6da22627b28b4b19bef57184d3167576" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 46401 } } peers { permanent_uuid: "f0dda022ce3140e08642813b191a3d19" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36913 } }
I20260806 21:14:42.651357 9327 heartbeater.cc:503] Master 127.16.178.62:37115 was elected leader, sending a full tablet report...
I20260806 21:14:42.651706 9549 ts_tablet_manager.cc:1428] T 93ab0a5ed55649caa16eaf0cffbf4465 P 1522aa3f9fe54b1d95e66ff625693695: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 21:14:42.651937 9526 heartbeater.cc:503] Master 127.16.178.62:37115 was elected leader, sending a full tablet report...
I20260806 21:14:42.652675 9551 ts_tablet_manager.cc:1428] T 93ab0a5ed55649caa16eaf0cffbf4465 P 6da22627b28b4b19bef57184d3167576: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260806 21:14:42.675045 9561 raft_consensus.cc:482] T 93ab0a5ed55649caa16eaf0cffbf4465 P 6da22627b28b4b19bef57184d3167576 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:14:42.675305 9561 raft_consensus.cc:504] T 93ab0a5ed55649caa16eaf0cffbf4465 P 6da22627b28b4b19bef57184d3167576 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1522aa3f9fe54b1d95e66ff625693695" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 37533 } } peers { permanent_uuid: "6da22627b28b4b19bef57184d3167576" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 46401 } } peers { permanent_uuid: "f0dda022ce3140e08642813b191a3d19" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36913 } }
I20260806 21:14:42.676695 9561 leader_election.cc:288] T 93ab0a5ed55649caa16eaf0cffbf4465 P 6da22627b28b4b19bef57184d3167576 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1522aa3f9fe54b1d95e66ff625693695 (127.16.178.2:37533), f0dda022ce3140e08642813b191a3d19 (127.16.178.1:36913)
I20260806 21:14:42.681218 9544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:42.677109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:40791 (local address 127.16.178.2:37533)
0806 21:14:42.677228 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:42.677232 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:42.677706 (+ 474us) server_negotiation.cc:407] Connection header received
0806 21:14:42.677797 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:42.677801 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:42.677857 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:42.677936 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:14:42.678202 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:42.678859 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:42.680256 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:42.680681 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:42.680798 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:42.680857 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:42.681019 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 21:14:42.681092 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 21:14:42.681241 9562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:42.676979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.2:37533 (local address 127.16.178.3:40791)
0806 21:14:42.677604 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:42.677618 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:14:42.677713 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:14:42.677974 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:14:42.677977 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:14:42.677995 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:14:42.678090 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:42.678095 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:42.678956 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:42.678959 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:42.680153 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:42.680163 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:42.680799 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:42.680802 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:42.680902 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:42.680925 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 21:14:42.680990 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 21:14:42.681064 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"mutex_wait_us":180,"thread_start_us":156,"threads_started":1}
I20260806 21:14:42.681288 9548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:42.677113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:50847 (local address 127.16.178.1:36913)
0806 21:14:42.677246 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:42.677251 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:42.677269 (+ 18us) server_negotiation.cc:407] Connection header received
0806 21:14:42.677386 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:42.677390 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:42.677449 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:42.677535 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:14:42.677853 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:42.678605 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:42.680284 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:42.680723 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:42.680844 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:42.680898 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:42.681110 (+ 212us) server_negotiation.cc:299] Negotiation successful
0806 21:14:42.681183 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 21:14:42.681284 9529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:42.676980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:36913 (local address 127.16.178.3:50847)
0806 21:14:42.677128 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:42.677146 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 21:14:42.677268 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:14:42.677578 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:14:42.677581 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:14:42.677602 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:14:42.677729 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:42.677737 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:42.678728 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:42.678731 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:42.680017 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:14:42.680029 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:14:42.680853 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:14:42.680857 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:14:42.680985 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:42.681012 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 21:14:42.681079 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 21:14:42.681154 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 21:14:42.681993 9255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93ab0a5ed55649caa16eaf0cffbf4465" candidate_uuid: "6da22627b28b4b19bef57184d3167576" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1522aa3f9fe54b1d95e66ff625693695" is_pre_election: true
I20260806 21:14:42.682004 9041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93ab0a5ed55649caa16eaf0cffbf4465" candidate_uuid: "6da22627b28b4b19bef57184d3167576" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0dda022ce3140e08642813b191a3d19" is_pre_election: true
I20260806 21:14:42.682317 9255 raft_consensus.cc:2454] T 93ab0a5ed55649caa16eaf0cffbf4465 P 1522aa3f9fe54b1d95e66ff625693695 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6da22627b28b4b19bef57184d3167576 in term 0.
I20260806 21:14:42.682315 9041 raft_consensus.cc:2454] T 93ab0a5ed55649caa16eaf0cffbf4465 P f0dda022ce3140e08642813b191a3d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6da22627b28b4b19bef57184d3167576 in term 0.
I20260806 21:14:42.682822 9362 leader_election.cc:302] T 93ab0a5ed55649caa16eaf0cffbf4465 P 6da22627b28b4b19bef57184d3167576 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1522aa3f9fe54b1d95e66ff625693695, 6da22627b28b4b19bef57184d3167576; no voters:
I20260806 21:14:42.683115 9561 raft_consensus.cc:2781] T 93ab0a5ed55649caa16eaf0cffbf4465 P 6da22627b28b4b19bef57184d3167576 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:14:42.683199 9561 raft_consensus.cc:482] T 93ab0a5ed55649caa16eaf0cffbf4465 P 6da22627b28b4b19bef57184d3167576 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:14:42.683228 9561 raft_consensus.cc:3037] T 93ab0a5ed55649caa16eaf0cffbf4465 P 6da22627b28b4b19bef57184d3167576 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:14:42.684329 9561 raft_consensus.cc:504] T 93ab0a5ed55649caa16eaf0cffbf4465 P 6da22627b28b4b19bef57184d3167576 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1522aa3f9fe54b1d95e66ff625693695" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 37533 } } peers { permanent_uuid: "6da22627b28b4b19bef57184d3167576" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 46401 } } peers { permanent_uuid: "f0dda022ce3140e08642813b191a3d19" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36913 } }
I20260806 21:14:42.684697 9561 leader_election.cc:288] T 93ab0a5ed55649caa16eaf0cffbf4465 P 6da22627b28b4b19bef57184d3167576 [CANDIDATE]: Term 1 election: Requested vote from peers 1522aa3f9fe54b1d95e66ff625693695 (127.16.178.2:37533), f0dda022ce3140e08642813b191a3d19 (127.16.178.1:36913)
I20260806 21:14:42.685079 9041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93ab0a5ed55649caa16eaf0cffbf4465" candidate_uuid: "6da22627b28b4b19bef57184d3167576" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0dda022ce3140e08642813b191a3d19"
I20260806 21:14:42.685079 9255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93ab0a5ed55649caa16eaf0cffbf4465" candidate_uuid: "6da22627b28b4b19bef57184d3167576" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1522aa3f9fe54b1d95e66ff625693695"
I20260806 21:14:42.685212 9041 raft_consensus.cc:3037] T 93ab0a5ed55649caa16eaf0cffbf4465 P f0dda022ce3140e08642813b191a3d19 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:14:42.685211 9255 raft_consensus.cc:3037] T 93ab0a5ed55649caa16eaf0cffbf4465 P 1522aa3f9fe54b1d95e66ff625693695 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:14:42.686494 9255 raft_consensus.cc:2454] T 93ab0a5ed55649caa16eaf0cffbf4465 P 1522aa3f9fe54b1d95e66ff625693695 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6da22627b28b4b19bef57184d3167576 in term 1.
I20260806 21:14:42.686494 9041 raft_consensus.cc:2454] T 93ab0a5ed55649caa16eaf0cffbf4465 P f0dda022ce3140e08642813b191a3d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6da22627b28b4b19bef57184d3167576 in term 1.
I20260806 21:14:42.686817 9362 leader_election.cc:302] T 93ab0a5ed55649caa16eaf0cffbf4465 P 6da22627b28b4b19bef57184d3167576 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1522aa3f9fe54b1d95e66ff625693695, 6da22627b28b4b19bef57184d3167576, f0dda022ce3140e08642813b191a3d19; no voters:
I20260806 21:14:42.687016 9561 raft_consensus.cc:2781] T 93ab0a5ed55649caa16eaf0cffbf4465 P 6da22627b28b4b19bef57184d3167576 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:14:42.687253 9561 raft_consensus.cc:686] T 93ab0a5ed55649caa16eaf0cffbf4465 P 6da22627b28b4b19bef57184d3167576 [term 1 LEADER]: Becoming Leader. State: Replica: 6da22627b28b4b19bef57184d3167576, State: Running, Role: LEADER
I20260806 21:14:42.687592 9561 consensus_queue.cc:237] T 93ab0a5ed55649caa16eaf0cffbf4465 P 6da22627b28b4b19bef57184d3167576 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1522aa3f9fe54b1d95e66ff625693695" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 37533 } } peers { permanent_uuid: "6da22627b28b4b19bef57184d3167576" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 46401 } } peers { permanent_uuid: "f0dda022ce3140e08642813b191a3d19" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36913 } }
I20260806 21:14:42.691008 8819 catalog_manager.cc:5495] T 93ab0a5ed55649caa16eaf0cffbf4465 P 6da22627b28b4b19bef57184d3167576 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6da22627b28b4b19bef57184d3167576 (127.16.178.3). New cstate: current_term: 1 leader_uuid: "6da22627b28b4b19bef57184d3167576" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1522aa3f9fe54b1d95e66ff625693695" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 37533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6da22627b28b4b19bef57184d3167576" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 46401 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f0dda022ce3140e08642813b191a3d19" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36913 } health_report { overall_health: UNKNOWN } } }
I20260806 21:14:42.703379 9547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:42.700152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45100 (local address 127.16.178.3:46401)
0806 21:14:42.700267 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 21:14:42.700271 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:14:42.700286 (+ 15us) server_negotiation.cc:407] Connection header received
0806 21:14:42.700376 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:14:42.700378 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:14:42.700430 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:14:42.700518 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:14:42.700722 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:42.701219 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:42.702018 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:14:42.702236 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:14:42.702317 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:14:42.702563 (+ 246us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:14:42.702993 (+ 430us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:14:42.703081 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 21:14:42.703198 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 21:14:42.703247 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 21:14:42.709628 9041 raft_consensus.cc:1261] T 93ab0a5ed55649caa16eaf0cffbf4465 P f0dda022ce3140e08642813b191a3d19 [term 1 FOLLOWER]: Refusing update from remote peer 6da22627b28b4b19bef57184d3167576: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:14:42.709628 9255 raft_consensus.cc:1261] T 93ab0a5ed55649caa16eaf0cffbf4465 P 1522aa3f9fe54b1d95e66ff625693695 [term 1 FOLLOWER]: Refusing update from remote peer 6da22627b28b4b19bef57184d3167576: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:14:42.710453 9561 consensus_queue.cc:1035] T 93ab0a5ed55649caa16eaf0cffbf4465 P 6da22627b28b4b19bef57184d3167576 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0dda022ce3140e08642813b191a3d19" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:14:42.710671 9564 consensus_queue.cc:1035] T 93ab0a5ed55649caa16eaf0cffbf4465 P 6da22627b28b4b19bef57184d3167576 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1522aa3f9fe54b1d95e66ff625693695" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 37533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:14:42.720270 9572 mvcc.cc:205] Tried to move back new op lower bound from 7315664415570132992 to 7315664415491399680. Current Snapshot: MvccSnapshot[applied={T|T < 7315664415570132992}]
I20260806 21:14:42.726354 8810 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51316:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:14:42.726526 8810 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51316:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:14:42.729081 8810 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7: Sending DeleteTablet for 3 replicas of tablet 93ab0a5ed55649caa16eaf0cffbf4465
I20260806 21:14:42.729887 9427 tablet_service.cc:1514] Processing DeleteTablet for tablet 93ab0a5ed55649caa16eaf0cffbf4465 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:14:42 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:45090
I20260806 21:14:42.729907 9223 tablet_service.cc:1514] Processing DeleteTablet for tablet 93ab0a5ed55649caa16eaf0cffbf4465 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:14:42 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:46306
I20260806 21:14:42.730118 9013 tablet_service.cc:1514] Processing DeleteTablet for tablet 93ab0a5ed55649caa16eaf0cffbf4465 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:14:42 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:51656
I20260806 21:14:42.730361 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 8913
I20260806 21:14:42.737910 9581 tablet_replica.cc:330] T 93ab0a5ed55649caa16eaf0cffbf4465 P 6da22627b28b4b19bef57184d3167576: stopping tablet replica
I20260806 21:14:42.738180 9582 tablet_replica.cc:330] T 93ab0a5ed55649caa16eaf0cffbf4465 P 1522aa3f9fe54b1d95e66ff625693695: stopping tablet replica
I20260806 21:14:42.738544 9582 raft_consensus.cc:2229] T 93ab0a5ed55649caa16eaf0cffbf4465 P 1522aa3f9fe54b1d95e66ff625693695 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:14:42.738852 9581 raft_consensus.cc:2229] T 93ab0a5ed55649caa16eaf0cffbf4465 P 6da22627b28b4b19bef57184d3167576 [term 1 LEADER]: Raft consensus shutting down.
I20260806 21:14:42.739262 9582 raft_consensus.cc:2258] T 93ab0a5ed55649caa16eaf0cffbf4465 P 1522aa3f9fe54b1d95e66ff625693695 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:14:42.739645 9581 raft_consensus.cc:2258] T 93ab0a5ed55649caa16eaf0cffbf4465 P 6da22627b28b4b19bef57184d3167576 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:14:42.744735 9582 ts_tablet_manager.cc:1905] T 93ab0a5ed55649caa16eaf0cffbf4465 P 1522aa3f9fe54b1d95e66ff625693695: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:14:42.747611 9582 ts_tablet_manager.cc:1918] T 93ab0a5ed55649caa16eaf0cffbf4465 P 1522aa3f9fe54b1d95e66ff625693695: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:14:42.747722 9582 log.cc:1196] T 93ab0a5ed55649caa16eaf0cffbf4465 P 1522aa3f9fe54b1d95e66ff625693695: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-1/wal/wals/93ab0a5ed55649caa16eaf0cffbf4465
I20260806 21:14:42.748101 9582 ts_tablet_manager.cc:1939] T 93ab0a5ed55649caa16eaf0cffbf4465 P 1522aa3f9fe54b1d95e66ff625693695: Deleting consensus metadata
I20260806 21:14:42.749037 9581 ts_tablet_manager.cc:1905] T 93ab0a5ed55649caa16eaf0cffbf4465 P 6da22627b28b4b19bef57184d3167576: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:14:42.749770 8797 catalog_manager.cc:4841] TS 1522aa3f9fe54b1d95e66ff625693695 (127.16.178.2:37533): tablet 93ab0a5ed55649caa16eaf0cffbf4465 (table test-table [id=2d56910231104bd5a074b5a8dabdbd35]) successfully deleted
I20260806 21:14:42.751865 9581 ts_tablet_manager.cc:1918] T 93ab0a5ed55649caa16eaf0cffbf4465 P 6da22627b28b4b19bef57184d3167576: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:14:42.751989 9581 log.cc:1196] T 93ab0a5ed55649caa16eaf0cffbf4465 P 6da22627b28b4b19bef57184d3167576: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-2/wal/wals/93ab0a5ed55649caa16eaf0cffbf4465
I20260806 21:14:42.752395 9581 ts_tablet_manager.cc:1939] T 93ab0a5ed55649caa16eaf0cffbf4465 P 6da22627b28b4b19bef57184d3167576: Deleting consensus metadata
W20260806 21:14:42.753140 8795 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.178.1:36913 (error 108)
I20260806 21:14:42.753441 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 9123
I20260806 21:14:42.753667 8796 catalog_manager.cc:4841] TS 6da22627b28b4b19bef57184d3167576 (127.16.178.3:46401): tablet 93ab0a5ed55649caa16eaf0cffbf4465 (table test-table [id=2d56910231104bd5a074b5a8dabdbd35]) successfully deleted
I20260806 21:14:42.754398 9540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:14:42.754007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:36913 (local address 127.0.0.1:51660)
0806 21:14:42.754219 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 21:14:42.754301 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.1:36913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
W20260806 21:14:42.754593 8795 catalog_manager.cc:4584] TS f0dda022ce3140e08642813b191a3d19 (127.16.178.1:36913): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 93ab0a5ed55649caa16eaf0cffbf4465: Network error: Client connection negotiation failed: client connection to 127.16.178.1:36913: connect: Connection refused (error 111)
I20260806 21:14:42.767851 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 9334
I20260806 21:14:42.779942 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 8766
I20260806 21:15:02.792732 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:37115
--webserver_interface=127.16.178.62
--webserver_port=44117
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:45599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:37115
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:15:02.909286 9586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:15:02.909600 9586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:15:02.909658 9586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:15:02.913887 9586 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 21:15:02.913959 9586 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 21:15:02.913982 9586 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 21:15:02.914000 9586 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:15:02.914024 9586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:15:02.914057 9586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:15:02.914086 9586 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:15:02.914114 9586 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:15:02.919711 9586 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:45599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:37115
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:37115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.178.62
--webserver_port=44117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9586
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:15:02.920948 9586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:15:02.922037 9586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:15:02.936187 9602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:02.936208 9597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:02.936278 9586 server_base.cc:1034] running on GCE node
W20260806 21:15:02.936198 9598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:02.937139 9586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:15:02.938329 9586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:15:02.939487 9586 hybrid_clock.cc:648] HybridClock initialized: now 1786050902939469 us; error 34 us; skew 500 ppm
Aug 06 21:15:02 dist-test-slave-lgr5 krb5kdc[8747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050902, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:15:02.951063 9586 init.cc:377] Logged in from keytab as kudu/127.16.178.62@KRBTEST.COM (short username kudu)
I20260806 21:15:02.952163 9586 webserver.cc:460] Webserver started at http://127.16.178.62:44117/ using document root <none> and password file <none>
I20260806 21:15:02.952800 9586 fs_manager.cc:359] Metadata directory not provided
I20260806 21:15:02.952885 9586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:15:02.957165 9586 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 21:15:02.959411 9614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:02.960798 9586 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:15:02.960927 9586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "82fdec997fa9475aa81fc138551e36e7"
format_stamp: "Formatted at 2026-08-06 21:14:41 on dist-test-slave-lgr5"
server_key: "e1354ebc86ec56fef3422989f77ade89"
server_key_iv: "437b13bb76db6a206a5f5e11e4f0132e"
server_key_version: "encryptionkey@0"
I20260806 21:15:02.961567 9586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:15:02.972378 9586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:15:02.975677 9586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:15:02.975873 9586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:15:02.983110 9586 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.62:37115
I20260806 21:15:02.983390 9696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.62:37115 every 8 connection(s)
I20260806 21:15:02.984314 9586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:15:02.988998 9697 sys_catalog.cc:263] Verifying existing consensus state
I20260806 21:15:02.990833 9697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7: Bootstrap starting.
I20260806 21:15:02.991233 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 9586
I20260806 21:15:02.991683 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.1:36913
--local_ip_for_outbound_sockets=127.16.178.1
--tserver_master_addrs=127.16.178.62:37115
--webserver_port=40463
--webserver_interface=127.16.178.1
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab
--principal=kudu/127.16.178.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:45599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 21:15:03.009210 9697 log.cc:824] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7: Log is configured to *not* fsync() on all Append() calls
I20260806 21:15:03.019884 9697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 21:15:03.020337 9697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7: Bootstrap complete.
I20260806 21:15:03.024520 9697 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fdec997fa9475aa81fc138551e36e7" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 37115 } }
I20260806 21:15:03.024942 9697 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82fdec997fa9475aa81fc138551e36e7, State: Initialized, Role: FOLLOWER
I20260806 21:15:03.025406 9697 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fdec997fa9475aa81fc138551e36e7" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 37115 } }
I20260806 21:15:03.025532 9697 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:15:03.025606 9697 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:15:03.025699 9697 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [term 1 FOLLOWER]: Advancing to term 2
I20260806 21:15:03.027982 9697 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fdec997fa9475aa81fc138551e36e7" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 37115 } }
I20260806 21:15:03.028446 9697 leader_election.cc:302] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82fdec997fa9475aa81fc138551e36e7; no voters:
I20260806 21:15:03.028872 9697 leader_election.cc:288] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 21:15:03.029033 9702 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 21:15:03.029390 9702 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [term 2 LEADER]: Becoming Leader. State: Replica: 82fdec997fa9475aa81fc138551e36e7, State: Running, Role: LEADER
I20260806 21:15:03.029898 9702 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fdec997fa9475aa81fc138551e36e7" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 37115 } }
I20260806 21:15:03.030265 9697 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:15:03.031389 9702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "82fdec997fa9475aa81fc138551e36e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fdec997fa9475aa81fc138551e36e7" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 37115 } } }
I20260806 21:15:03.031606 9702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [sys.catalog]: This master's current role is: LEADER
I20260806 21:15:03.031857 9702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82fdec997fa9475aa81fc138551e36e7. Latest consensus state: current_term: 2 leader_uuid: "82fdec997fa9475aa81fc138551e36e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fdec997fa9475aa81fc138551e36e7" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 37115 } } }
I20260806 21:15:03.032022 9702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [sys.catalog]: This master's current role is: LEADER
I20260806 21:15:03.032361 9716 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:15:03.038950 9716 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:15:03.039700 9716 catalog_manager.cc:1259] Loaded cluster ID: 0ac10d4413944f2293d8ec7cb6b3a3d3
I20260806 21:15:03.039808 9716 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:15:03.042470 9716 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:15:03.043201 9716 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7: Loaded TSK: 0
I20260806 21:15:03.059454 9716 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7: Generated new TSK 1
I20260806 21:15:03.059760 9716 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 21:15:03.118841 9699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:15:03.119108 9699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:15:03.119155 9699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:15:03.123291 9699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:15:03.123359 9699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:15:03.123420 9699 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.1
I20260806 21:15:03.129173 9699 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:45599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab
--principal=kudu/127.16.178.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.1:36913
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.178.1
--webserver_port=40463
--tserver_master_addrs=127.16.178.62:37115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9699
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.1
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:15:03.130271 9699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:15:03.131347 9699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:15:03.142081 9741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:03.142081 9742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:03.142381 9699 server_base.cc:1034] running on GCE node
W20260806 21:15:03.142604 9744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:03.143049 9699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:15:03.143765 9699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:15:03.144963 9699 hybrid_clock.cc:648] HybridClock initialized: now 1786050903144933 us; error 46 us; skew 500 ppm
Aug 06 21:15:03 dist-test-slave-lgr5 krb5kdc[8747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050903, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:15:03.148160 9699 init.cc:377] Logged in from keytab as kudu/127.16.178.1@KRBTEST.COM (short username kudu)
I20260806 21:15:03.149348 9699 webserver.cc:460] Webserver started at http://127.16.178.1:40463/ using document root <none> and password file <none>
I20260806 21:15:03.149987 9699 fs_manager.cc:359] Metadata directory not provided
I20260806 21:15:03.150094 9699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:15:03.154709 9699 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 21:15:03.157074 9755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:03.158378 9699 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:15:03.158511 9699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-0/wal
uuid: "f0dda022ce3140e08642813b191a3d19"
format_stamp: "Formatted at 2026-08-06 21:14:42 on dist-test-slave-lgr5"
server_key: "3ef4935787c7a2b933e6df7f4350afb1"
server_key_iv: "a8e0caa24230334a4dbd9d7a9fbfa49a"
server_key_version: "encryptionkey@0"
I20260806 21:15:03.159118 9699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:15:03.178366 9699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:15:03.181699 9699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:15:03.181890 9699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:15:03.182660 9699 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:15:03.183825 9765 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 21:15:03.188287 9699 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 21:15:03.188412 9699 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260806 21:15:03.188506 9699 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 21:15:03.190605 9699 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 21:15:03.190703 9699 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260806 21:15:03.191067 9765 tablet_bootstrap.cc:492] T 93ab0a5ed55649caa16eaf0cffbf4465 P f0dda022ce3140e08642813b191a3d19: Bootstrap starting.
I20260806 21:15:03.202359 9699 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.1:36913
I20260806 21:15:03.202581 9926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.1:36913 every 8 connection(s)
I20260806 21:15:03.203439 9699 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
I20260806 21:15:03.211210 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 9699
I20260806 21:15:03.211717 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.2:37533
--local_ip_for_outbound_sockets=127.16.178.2
--tserver_master_addrs=127.16.178.62:37115
--webserver_port=44737
--webserver_interface=127.16.178.2
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab
--principal=kudu/127.16.178.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:45599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 21:15:03.212646 9765 log.cc:824] T 93ab0a5ed55649caa16eaf0cffbf4465 P f0dda022ce3140e08642813b191a3d19: Log is configured to *not* fsync() on all Append() calls
Aug 06 21:15:03 dist-test-slave-lgr5 krb5kdc[8747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050903, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:15:03.221854 9765 tablet_bootstrap.cc:492] T 93ab0a5ed55649caa16eaf0cffbf4465 P f0dda022ce3140e08642813b191a3d19: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 21:15:03.222054 9935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:03.205758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.1:56407)
0806 21:15:03.206291 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:03.206335 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:03.207159 (+ 824us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:03.208304 (+ 1145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:03.208312 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:03.208802 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:15:03.209273 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:03.209288 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:03.210597 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:03.210600 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:03.211803 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:03.211817 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:03.212997 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:03.213008 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:03.213222 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:03.213905 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:15:03.213927 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:15:03.215662 (+ 1735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:15:03.218305 (+ 2643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:15:03.218311 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:15:03.218323 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:15:03.218695 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:15:03.219181 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:15:03.219184 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:15:03.219186 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:15:03.219364 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:15:03.220190 (+ 826us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:15:03.220196 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:15:03.220505 (+ 309us) client_negotiation.cc:769] Sending connection context
0806 21:15:03.220705 (+ 200us) client_negotiation.cc:240] Negotiation successful
0806 21:15:03.221058 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":165,"threads_started":1}
I20260806 21:15:03.222174 9933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:03.205636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:56407 (local address 127.16.178.62:37115)
0806 21:15:03.206023 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:03.206039 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:03.206480 (+ 441us) server_negotiation.cc:407] Connection header received
0806 21:15:03.207573 (+ 1093us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:03.207583 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:03.207871 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:03.208208 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:15:03.209452 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:03.210437 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:03.211958 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:03.212671 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:03.212836 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:03.215899 (+ 3063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:15:03.215905 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:15:03.215919 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:15:03.215952 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:15:03.218152 (+ 2200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:03.218866 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:03.218874 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:03.218880 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:03.219069 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:03.219470 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:03.219473 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:03.219475 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:03.220028 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:15:03.220207 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:03.220924 (+ 717us) server_negotiation.cc:299] Negotiation successful
0806 21:15:03.221277 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":113,"threads_started":1}
I20260806 21:15:03.222357 9765 tablet_bootstrap.cc:492] T 93ab0a5ed55649caa16eaf0cffbf4465 P f0dda022ce3140e08642813b191a3d19: Bootstrap complete.
I20260806 21:15:03.223714 9765 ts_tablet_manager.cc:1397] T 93ab0a5ed55649caa16eaf0cffbf4465 P f0dda022ce3140e08642813b191a3d19: Time spent bootstrapping tablet: real 0.033s user 0.022s sys 0.007s
I20260806 21:15:03.225133 9929 heartbeater.cc:347] Connected to a master server at 127.16.178.62:37115
I20260806 21:15:03.225385 9929 heartbeater.cc:464] Registering TS with master...
I20260806 21:15:03.225986 9929 heartbeater.cc:511] Master 127.16.178.62:37115 requested a full tablet report, sending...
I20260806 21:15:03.226526 9765 raft_consensus.cc:348] T 93ab0a5ed55649caa16eaf0cffbf4465 P f0dda022ce3140e08642813b191a3d19 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1522aa3f9fe54b1d95e66ff625693695" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 37533 } } peers { permanent_uuid: "6da22627b28b4b19bef57184d3167576" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 46401 } } peers { permanent_uuid: "f0dda022ce3140e08642813b191a3d19" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36913 } }
I20260806 21:15:03.227074 9765 raft_consensus.cc:729] T 93ab0a5ed55649caa16eaf0cffbf4465 P f0dda022ce3140e08642813b191a3d19 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0dda022ce3140e08642813b191a3d19, State: Initialized, Role: FOLLOWER
I20260806 21:15:03.227582 9765 consensus_queue.cc:260] T 93ab0a5ed55649caa16eaf0cffbf4465 P f0dda022ce3140e08642813b191a3d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1522aa3f9fe54b1d95e66ff625693695" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 37533 } } peers { permanent_uuid: "6da22627b28b4b19bef57184d3167576" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 46401 } } peers { permanent_uuid: "f0dda022ce3140e08642813b191a3d19" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36913 } }
I20260806 21:15:03.228725 9765 ts_tablet_manager.cc:1428] T 93ab0a5ed55649caa16eaf0cffbf4465 P f0dda022ce3140e08642813b191a3d19: Time spent starting tablet: real 0.005s user 0.007s sys 0.001s
I20260806 21:15:03.229153 9642 ts_manager.cc:194] Registered new tserver with Master: f0dda022ce3140e08642813b191a3d19 (127.16.178.1:36913)
I20260806 21:15:03.232784 9642 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.1@KRBTEST.COM'} at 127.16.178.1:56407
I20260806 21:15:03.235574 9929 heartbeater.cc:503] Master 127.16.178.62:37115 was elected leader, sending a full tablet report...
Aug 06 21:15:03 dist-test-slave-lgr5 krb5kdc[8747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050902, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for kudu/127.16.178.1@KRBTEST.COM
I20260806 21:15:03.243599 9940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:03.232305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:36913 (local address 127.0.0.1:59882)
0806 21:15:03.232881 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:03.232929 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:03.233078 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:03.233845 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:03.233852 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:03.234326 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:15:03.234473 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:03.234482 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:03.235824 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:03.235828 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:03.236677 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:03.236688 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:03.237311 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:03.237315 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:03.237439 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:03.238072 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:15:03.238085 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:15:03.239810 (+ 1725us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:15:03.241756 (+ 1946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:15:03.241768 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:15:03.241775 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:15:03.242018 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:15:03.242322 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:15:03.242325 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:15:03.242327 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:15:03.242410 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:15:03.242892 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:15:03.242898 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:15:03.243026 (+ 128us) client_negotiation.cc:769] Sending connection context
0806 21:15:03.243150 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 21:15:03.243379 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":115,"threads_started":1}
I20260806 21:15:03.243599 9941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:03.232348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59882 (local address 127.16.178.1:36913)
0806 21:15:03.233358 (+ 1010us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:03.233369 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:03.233392 (+ 23us) server_negotiation.cc:407] Connection header received
0806 21:15:03.233466 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:03.233471 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:03.233679 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:03.233823 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:15:03.234624 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:03.235673 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:03.236802 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:03.237178 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:03.237284 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:03.240002 (+ 2718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:15:03.240015 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:15:03.240028 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:15:03.240060 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:15:03.241631 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:03.242140 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:03.242145 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:03.242147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:03.242230 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:03.242495 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:03.242499 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:03.242501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:03.242774 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:15:03.242895 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:03.243255 (+ 360us) server_negotiation.cc:299] Negotiation successful
0806 21:15:03.243424 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":908,"thread_start_us":118,"threads_started":1}
I20260806 21:15:03.244910 9825 tablet_service.cc:1514] Processing DeleteTablet for tablet 93ab0a5ed55649caa16eaf0cffbf4465 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:14:42 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:59882
I20260806 21:15:03.245760 9942 tablet_replica.cc:330] stopping tablet replica
I20260806 21:15:03.246078 9942 raft_consensus.cc:2229] T 93ab0a5ed55649caa16eaf0cffbf4465 P f0dda022ce3140e08642813b191a3d19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:15:03.246326 9942 raft_consensus.cc:2258] T 93ab0a5ed55649caa16eaf0cffbf4465 P f0dda022ce3140e08642813b191a3d19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:15:03.247318 9942 ts_tablet_manager.cc:1905] T 93ab0a5ed55649caa16eaf0cffbf4465 P f0dda022ce3140e08642813b191a3d19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:15:03.250952 9942 ts_tablet_manager.cc:1918] T 93ab0a5ed55649caa16eaf0cffbf4465 P f0dda022ce3140e08642813b191a3d19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:15:03.251050 9942 log.cc:1196] T 93ab0a5ed55649caa16eaf0cffbf4465 P f0dda022ce3140e08642813b191a3d19: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-0/wal/wals/93ab0a5ed55649caa16eaf0cffbf4465
I20260806 21:15:03.251272 9942 ts_tablet_manager.cc:1939] T 93ab0a5ed55649caa16eaf0cffbf4465 P f0dda022ce3140e08642813b191a3d19: Deleting consensus metadata
I20260806 21:15:03.253110 9619 catalog_manager.cc:4841] TS f0dda022ce3140e08642813b191a3d19 (127.16.178.1:36913): tablet 93ab0a5ed55649caa16eaf0cffbf4465 (table test-table [id=2d56910231104bd5a074b5a8dabdbd35]) successfully deleted
W20260806 21:15:03.335896 9938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:15:03.336277 9938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:15:03.336330 9938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:15:03.340631 9938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:15:03.340713 9938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:15:03.340778 9938 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.2
I20260806 21:15:03.346532 9938 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:45599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab
--principal=kudu/127.16.178.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.2:37533
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.178.2
--webserver_port=44737
--tserver_master_addrs=127.16.178.62:37115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9938
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.2
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:15:03.347756 9938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:15:03.348914 9938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:15:03.359347 9951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:03.359560 9938 server_base.cc:1034] running on GCE node
W20260806 21:15:03.359328 9949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:03.359328 9948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:03.360162 9938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:15:03.360946 9938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:15:03.362115 9938 hybrid_clock.cc:648] HybridClock initialized: now 1786050903362093 us; error 39 us; skew 500 ppm
Aug 06 21:15:03 dist-test-slave-lgr5 krb5kdc[8747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050903, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:15:03.365268 9938 init.cc:377] Logged in from keytab as kudu/127.16.178.2@KRBTEST.COM (short username kudu)
I20260806 21:15:03.366287 9938 webserver.cc:460] Webserver started at http://127.16.178.2:44737/ using document root <none> and password file <none>
I20260806 21:15:03.366882 9938 fs_manager.cc:359] Metadata directory not provided
I20260806 21:15:03.366981 9938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:15:03.371227 9938 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.002s
I20260806 21:15:03.373558 9961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:03.374898 9938 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 21:15:03.375016 9938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-1/wal
uuid: "1522aa3f9fe54b1d95e66ff625693695"
format_stamp: "Formatted at 2026-08-06 21:14:42 on dist-test-slave-lgr5"
server_key: "3733ff686be7eb1cfb4140b162b931f4"
server_key_iv: "84e3ed641f1ef6253e1969d8d22d01b0"
server_key_version: "encryptionkey@0"
I20260806 21:15:03.375643 9938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:15:03.391256 9938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:15:03.394613 9938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:15:03.394806 9938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:15:03.395485 9938 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:15:03.396414 9938 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:15:03.396466 9938 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:03.396512 9938 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:15:03.396543 9938 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:03.405382 9938 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.2:37533
I20260806 21:15:03.405412 10128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.2:37533 every 8 connection(s)
I20260806 21:15:03.406454 9938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
I20260806 21:15:03.411963 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 9938
I20260806 21:15:03.412403 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.3:46401
--local_ip_for_outbound_sockets=127.16.178.3
--tserver_master_addrs=127.16.178.62:37115
--webserver_port=44075
--webserver_interface=127.16.178.3
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab
--principal=kudu/127.16.178.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:45599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 21:15:03 dist-test-slave-lgr5 krb5kdc[8747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050903, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.2@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:15:03.421317 9933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:03.408341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:60711 (local address 127.16.178.62:37115)
0806 21:15:03.408519 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:03.408522 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:03.409364 (+ 842us) server_negotiation.cc:407] Connection header received
0806 21:15:03.410701 (+ 1337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:03.410704 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:03.410767 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:03.410842 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:15:03.412073 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:03.412695 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:03.413862 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:03.414178 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:03.414287 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:03.417057 (+ 2770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:15:03.417062 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:15:03.417065 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:15:03.417098 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:15:03.418713 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:03.419301 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:03.419305 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:03.419307 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:03.419408 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:03.419805 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:03.419808 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:03.419810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:03.420122 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:15:03.420227 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:03.420963 (+ 736us) server_negotiation.cc:299] Negotiation successful
0806 21:15:03.421126 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 21:15:03.422220 10131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:03.408737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.2:60711)
0806 21:15:03.409210 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:03.409254 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:03.410483 (+ 1229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:03.411043 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:03.411050 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:03.411431 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:15:03.411918 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:03.411933 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:03.412830 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:03.412833 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:03.413739 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:03.413750 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:03.414353 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:03.414356 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:03.414539 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:03.415157 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:15:03.415173 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:15:03.416856 (+ 1683us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:15:03.418862 (+ 2006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:15:03.418868 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:15:03.418882 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:15:03.419194 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:15:03.419534 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:15:03.419536 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:15:03.419538 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:15:03.419695 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:15:03.420250 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:15:03.420256 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:15:03.420667 (+ 411us) client_negotiation.cc:769] Sending connection context
0806 21:15:03.420851 (+ 184us) client_negotiation.cc:240] Negotiation successful
0806 21:15:03.421151 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":123,"threads_started":1}
I20260806 21:15:03.423581 10129 heartbeater.cc:347] Connected to a master server at 127.16.178.62:37115
I20260806 21:15:03.423837 10129 heartbeater.cc:464] Registering TS with master...
I20260806 21:15:03.424412 10129 heartbeater.cc:511] Master 127.16.178.62:37115 requested a full tablet report, sending...
I20260806 21:15:03.425565 9642 ts_manager.cc:194] Registered new tserver with Master: 1522aa3f9fe54b1d95e66ff625693695 (127.16.178.2:37533)
I20260806 21:15:03.426173 9642 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.2@KRBTEST.COM'} at 127.16.178.2:60711
W20260806 21:15:03.536926 10134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:15:03.537241 10134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:15:03.537325 10134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:15:03.541643 10134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:15:03.541715 10134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:15:03.541795 10134 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.3
I20260806 21:15:03.547405 10134 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:45599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab
--principal=kudu/127.16.178.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.3:46401
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.178.3
--webserver_port=44075
--tserver_master_addrs=127.16.178.62:37115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.3
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:15:03.548657 10134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:15:03.549785 10134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:15:03.560120 10142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:03.560144 10139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:03.560220 10134 server_base.cc:1034] running on GCE node
W20260806 21:15:03.560117 10140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:03.560832 10134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:15:03.561506 10134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:15:03.562661 10134 hybrid_clock.cc:648] HybridClock initialized: now 1786050903562638 us; error 35 us; skew 500 ppm
Aug 06 21:15:03 dist-test-slave-lgr5 krb5kdc[8747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050903, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:15:03.565542 10134 init.cc:377] Logged in from keytab as kudu/127.16.178.3@KRBTEST.COM (short username kudu)
I20260806 21:15:03.566520 10134 webserver.cc:460] Webserver started at http://127.16.178.3:44075/ using document root <none> and password file <none>
I20260806 21:15:03.567111 10134 fs_manager.cc:359] Metadata directory not provided
I20260806 21:15:03.567196 10134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:15:03.571424 10134 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:15:03.573707 10152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:03.575150 10134 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 21:15:03.575284 10134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-2/wal
uuid: "6da22627b28b4b19bef57184d3167576"
format_stamp: "Formatted at 2026-08-06 21:14:42 on dist-test-slave-lgr5"
server_key: "35923fcda49e926fa5e11018e8e2146a"
server_key_iv: "6b2e9dc722f6d970b1a5caa219436a04"
server_key_version: "encryptionkey@0"
I20260806 21:15:03.575947 10134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:15:03.601241 10134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:15:03.604518 10134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:15:03.604683 10134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:15:03.605261 10134 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:15:03.606088 10134 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:15:03.606140 10134 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:03.606179 10134 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:15:03.606200 10134 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:03.615155 10134 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.3:46401
I20260806 21:15:03.615182 10318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.3:46401 every 8 connection(s)
I20260806 21:15:03.616194 10134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
I20260806 21:15:03.621028 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 10134
Aug 06 21:15:03 dist-test-slave-lgr5 krb5kdc[8747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050903, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.3@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:15:03.629953 10323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:03.621646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40258 (local address 127.16.178.62:37115)
0806 21:15:03.621993 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:03.621997 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:03.622012 (+ 15us) server_negotiation.cc:407] Connection header received
0806 21:15:03.622128 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:03.622131 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:03.622172 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:03.622258 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:15:03.623137 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:03.623759 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:03.624837 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:03.625125 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:03.625211 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:03.626480 (+ 1269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:15:03.626484 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:15:03.626486 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:15:03.626517 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:15:03.628270 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:03.628755 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:03.628757 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:03.628758 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:03.628832 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:03.629109 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:03.629112 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:03.629114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:03.629304 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:15:03.629462 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:03.629665 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 21:15:03.629790 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":85,"threads_started":1}
I20260806 21:15:03.630825 9933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:03.617950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:58501 (local address 127.16.178.62:37115)
0806 21:15:03.618082 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:03.618086 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:03.618899 (+ 813us) server_negotiation.cc:407] Connection header received
0806 21:15:03.620188 (+ 1289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:03.620191 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:03.620253 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:03.620329 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:15:03.621591 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:03.622312 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:03.623759 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:03.624029 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:03.624152 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:03.626978 (+ 2826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:15:03.626982 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:15:03.626985 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:15:03.627041 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:15:03.628457 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:03.629011 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:03.629014 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:03.629015 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:03.629085 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:03.629460 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:03.629463 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:03.629465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:03.629655 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:15:03.629744 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:03.630592 (+ 848us) server_negotiation.cc:299] Negotiation successful
0806 21:15:03.630698 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 21:15:03.632205 10321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:03.618280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.3:58501)
0806 21:15:03.618750 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:03.618791 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:03.619948 (+ 1157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:03.620545 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:03.620552 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:03.620908 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:15:03.621429 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:03.621444 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:03.622436 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:03.622440 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:03.623643 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:03.623654 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:03.624195 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:03.624199 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:03.624398 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:03.625081 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:15:03.625097 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:15:03.626805 (+ 1708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:15:03.628559 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:15:03.628566 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:15:03.628582 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:15:03.628912 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:15:03.629173 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:15:03.629175 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:15:03.629177 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:15:03.629355 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:15:03.629777 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:15:03.629784 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:15:03.630107 (+ 323us) client_negotiation.cc:769] Sending connection context
0806 21:15:03.630581 (+ 474us) client_negotiation.cc:240] Negotiation successful
0806 21:15:03.631007 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":106,"threads_started":1}
I20260806 21:15:03.633390 10319 heartbeater.cc:347] Connected to a master server at 127.16.178.62:37115
I20260806 21:15:03.633646 10319 heartbeater.cc:464] Registering TS with master...
I20260806 21:15:03.634194 10319 heartbeater.cc:511] Master 127.16.178.62:37115 requested a full tablet report, sending...
I20260806 21:15:03.635300 9642 ts_manager.cc:194] Registered new tserver with Master: 6da22627b28b4b19bef57184d3167576 (127.16.178.3:46401)
I20260806 21:15:03.635910 9642 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.3@KRBTEST.COM'} at 127.16.178.3:58501
I20260806 21:15:03.644698 17096 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 21:15:03.649470 9933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:03.645535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40272 (local address 127.16.178.62:37115)
0806 21:15:03.645704 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:03.645708 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:03.645869 (+ 161us) server_negotiation.cc:407] Connection header received
0806 21:15:03.646063 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:03.646066 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:03.646189 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:03.646289 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:15:03.646597 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:03.647168 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:03.648121 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:03.648349 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:03.648456 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:03.648834 (+ 378us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:15:03.649019 (+ 185us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0806 21:15:03.649311 (+ 292us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40272: token expired
Metrics: {"server-negotiator.queue_time_us":63}
W20260806 21:15:03.649566 9933 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40272: token expired
W20260806 21:15:03.649920 9532 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.16.178.62:37115)
I20260806 21:15:03.657197 9933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:03.650328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40274 (local address 127.16.178.62:37115)
0806 21:15:03.650447 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:03.650451 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:03.650513 (+ 62us) server_negotiation.cc:407] Connection header received
0806 21:15:03.650644 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:03.650647 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:03.650692 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:03.650757 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:15:03.651379 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:03.651875 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:03.652739 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:03.652957 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:03.653043 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:03.654332 (+ 1289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:15:03.654336 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:15:03.654338 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:15:03.654366 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:15:03.655655 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:03.656037 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:03.656041 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:03.656043 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:03.656154 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:03.656407 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:03.656411 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:03.656413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:03.656628 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:15:03.656759 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:03.656979 (+ 220us) server_negotiation.cc:299] Negotiation successful
0806 21:15:03.657076 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 21:15:03.659864 9642 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40274:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:15:03.663491 9642 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:15:03.672652 9825 tablet_service.cc:1467] Processing CreateTablet for tablet 2b5f64c69b4245de92fc32b1a711faa7 (DEFAULT_TABLE table=test-table [id=9b3180249a0145abbed68c33c935aa45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:15:03.673269 9825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b5f64c69b4245de92fc32b1a711faa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:15:03.676195 9765 tablet_bootstrap.cc:492] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19: Bootstrap starting.
I20260806 21:15:03.676580 9940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:03.670942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.3:46401 (local address 127.0.0.1:53226)
0806 21:15:03.671118 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:03.671135 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:03.671223 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:03.672194 (+ 971us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:03.672198 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:03.672215 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:15:03.672330 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:03.672337 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:03.673807 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:03.673811 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:03.675085 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:03.675098 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:03.676152 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:03.676156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:03.676255 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:03.676277 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 21:15:03.676332 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 21:15:03.676419 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"mutex_wait_us":42}
I20260806 21:15:03.677151 10334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:03.671354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40678 (local address 127.16.178.2:37533)
0806 21:15:03.671875 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:03.671886 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:03.671908 (+ 22us) server_negotiation.cc:407] Connection header received
0806 21:15:03.671982 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:03.671988 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:03.672224 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:03.672368 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:15:03.672633 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:03.673853 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:03.674991 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:03.675816 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:03.675932 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:03.676143 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:03.676880 (+ 737us) server_negotiation.cc:299] Negotiation successful
0806 21:15:03.677003 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":172,"threads_started":1}
I20260806 21:15:03.677167 9765 tablet_bootstrap.cc:654] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19: Neither blocks nor log segments found. Creating new log.
I20260806 21:15:03.677380 10331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:03.671187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53226 (local address 127.16.178.3:46401)
0806 21:15:03.671661 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:03.671675 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:03.671701 (+ 26us) server_negotiation.cc:407] Connection header received
0806 21:15:03.671791 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:03.671797 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:03.672029 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:03.672216 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:15:03.672462 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:03.673679 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:03.675222 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:03.675988 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:03.676855 (+ 867us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:03.677016 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:03.677068 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 21:15:03.677151 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":113,"threads_started":1}
I20260806 21:15:03.677520 10333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:03.671172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.2:37533 (local address 127.0.0.1:40678)
0806 21:15:03.671451 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:03.671586 (+ 135us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:03.671678 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:03.672412 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:03.672415 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:03.672433 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:15:03.672520 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:03.672526 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:03.673964 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:03.673967 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:03.674865 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:03.674876 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:03.675965 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:03.675969 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:03.676058 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:03.676161 (+ 103us) client_negotiation.cc:769] Sending connection context
0806 21:15:03.676600 (+ 439us) client_negotiation.cc:240] Negotiation successful
0806 21:15:03.677373 (+ 773us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"mutex_wait_us":23,"thread_start_us":137,"threads_started":1}
I20260806 21:15:03.678714 9765 tablet_bootstrap.cc:492] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19: No bootstrap required, opened a new log
I20260806 21:15:03.678804 9765 ts_tablet_manager.cc:1397] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 21:15:03.679268 9765 raft_consensus.cc:348] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6da22627b28b4b19bef57184d3167576" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 46401 } } peers { permanent_uuid: "1522aa3f9fe54b1d95e66ff625693695" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 37533 } } peers { permanent_uuid: "f0dda022ce3140e08642813b191a3d19" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36913 } }
I20260806 21:15:03.679404 9765 raft_consensus.cc:374] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:15:03.679455 9765 raft_consensus.cc:729] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0dda022ce3140e08642813b191a3d19, State: Initialized, Role: FOLLOWER
I20260806 21:15:03.679838 9765 consensus_queue.cc:260] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6da22627b28b4b19bef57184d3167576" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 46401 } } peers { permanent_uuid: "1522aa3f9fe54b1d95e66ff625693695" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 37533 } } peers { permanent_uuid: "f0dda022ce3140e08642813b191a3d19" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36913 } }
I20260806 21:15:03.680711 9765 ts_tablet_manager.cc:1428] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 21:15:03.682132 10032 tablet_service.cc:1467] Processing CreateTablet for tablet 2b5f64c69b4245de92fc32b1a711faa7 (DEFAULT_TABLE table=test-table [id=9b3180249a0145abbed68c33c935aa45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:15:03.683264 10032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b5f64c69b4245de92fc32b1a711faa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:15:03.684768 10220 tablet_service.cc:1467] Processing CreateTablet for tablet 2b5f64c69b4245de92fc32b1a711faa7 (DEFAULT_TABLE table=test-table [id=9b3180249a0145abbed68c33c935aa45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:15:03.685845 10220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b5f64c69b4245de92fc32b1a711faa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:15:03.690397 10338 tablet_bootstrap.cc:492] T 2b5f64c69b4245de92fc32b1a711faa7 P 1522aa3f9fe54b1d95e66ff625693695: Bootstrap starting.
I20260806 21:15:03.692809 10340 tablet_bootstrap.cc:492] T 2b5f64c69b4245de92fc32b1a711faa7 P 6da22627b28b4b19bef57184d3167576: Bootstrap starting.
I20260806 21:15:03.693005 10338 tablet_bootstrap.cc:654] T 2b5f64c69b4245de92fc32b1a711faa7 P 1522aa3f9fe54b1d95e66ff625693695: Neither blocks nor log segments found. Creating new log.
I20260806 21:15:03.693972 10338 log.cc:824] T 2b5f64c69b4245de92fc32b1a711faa7 P 1522aa3f9fe54b1d95e66ff625693695: Log is configured to *not* fsync() on all Append() calls
I20260806 21:15:03.695488 10340 tablet_bootstrap.cc:654] T 2b5f64c69b4245de92fc32b1a711faa7 P 6da22627b28b4b19bef57184d3167576: Neither blocks nor log segments found. Creating new log.
I20260806 21:15:03.696385 10338 tablet_bootstrap.cc:492] T 2b5f64c69b4245de92fc32b1a711faa7 P 1522aa3f9fe54b1d95e66ff625693695: No bootstrap required, opened a new log
I20260806 21:15:03.696393 10340 log.cc:824] T 2b5f64c69b4245de92fc32b1a711faa7 P 6da22627b28b4b19bef57184d3167576: Log is configured to *not* fsync() on all Append() calls
I20260806 21:15:03.696609 10338 ts_tablet_manager.cc:1397] T 2b5f64c69b4245de92fc32b1a711faa7 P 1522aa3f9fe54b1d95e66ff625693695: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260806 21:15:03.698544 10340 tablet_bootstrap.cc:492] T 2b5f64c69b4245de92fc32b1a711faa7 P 6da22627b28b4b19bef57184d3167576: No bootstrap required, opened a new log
I20260806 21:15:03.698768 10340 ts_tablet_manager.cc:1397] T 2b5f64c69b4245de92fc32b1a711faa7 P 6da22627b28b4b19bef57184d3167576: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 21:15:03.700598 10338 raft_consensus.cc:348] T 2b5f64c69b4245de92fc32b1a711faa7 P 1522aa3f9fe54b1d95e66ff625693695 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6da22627b28b4b19bef57184d3167576" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 46401 } } peers { permanent_uuid: "1522aa3f9fe54b1d95e66ff625693695" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 37533 } } peers { permanent_uuid: "f0dda022ce3140e08642813b191a3d19" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36913 } }
I20260806 21:15:03.700945 10338 raft_consensus.cc:374] T 2b5f64c69b4245de92fc32b1a711faa7 P 1522aa3f9fe54b1d95e66ff625693695 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:15:03.701009 10338 raft_consensus.cc:729] T 2b5f64c69b4245de92fc32b1a711faa7 P 1522aa3f9fe54b1d95e66ff625693695 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1522aa3f9fe54b1d95e66ff625693695, State: Initialized, Role: FOLLOWER
I20260806 21:15:03.701558 10338 consensus_queue.cc:260] T 2b5f64c69b4245de92fc32b1a711faa7 P 1522aa3f9fe54b1d95e66ff625693695 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6da22627b28b4b19bef57184d3167576" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 46401 } } peers { permanent_uuid: "1522aa3f9fe54b1d95e66ff625693695" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 37533 } } peers { permanent_uuid: "f0dda022ce3140e08642813b191a3d19" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36913 } }
I20260806 21:15:03.702385 10129 heartbeater.cc:503] Master 127.16.178.62:37115 was elected leader, sending a full tablet report...
I20260806 21:15:03.702817 10338 ts_tablet_manager.cc:1428] T 2b5f64c69b4245de92fc32b1a711faa7 P 1522aa3f9fe54b1d95e66ff625693695: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 21:15:03.702764 10340 raft_consensus.cc:348] T 2b5f64c69b4245de92fc32b1a711faa7 P 6da22627b28b4b19bef57184d3167576 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6da22627b28b4b19bef57184d3167576" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 46401 } } peers { permanent_uuid: "1522aa3f9fe54b1d95e66ff625693695" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 37533 } } peers { permanent_uuid: "f0dda022ce3140e08642813b191a3d19" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36913 } }
I20260806 21:15:03.703083 10340 raft_consensus.cc:374] T 2b5f64c69b4245de92fc32b1a711faa7 P 6da22627b28b4b19bef57184d3167576 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:15:03.703150 10340 raft_consensus.cc:729] T 2b5f64c69b4245de92fc32b1a711faa7 P 6da22627b28b4b19bef57184d3167576 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6da22627b28b4b19bef57184d3167576, State: Initialized, Role: FOLLOWER
I20260806 21:15:03.703644 10340 consensus_queue.cc:260] T 2b5f64c69b4245de92fc32b1a711faa7 P 6da22627b28b4b19bef57184d3167576 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6da22627b28b4b19bef57184d3167576" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 46401 } } peers { permanent_uuid: "1522aa3f9fe54b1d95e66ff625693695" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 37533 } } peers { permanent_uuid: "f0dda022ce3140e08642813b191a3d19" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36913 } }
I20260806 21:15:03.704473 10319 heartbeater.cc:503] Master 127.16.178.62:37115 was elected leader, sending a full tablet report...
W20260806 21:15:03.704802 9930 tablet.cc:2369] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 21:15:03.705415 10340 ts_tablet_manager.cc:1428] T 2b5f64c69b4245de92fc32b1a711faa7 P 6da22627b28b4b19bef57184d3167576: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260806 21:15:03.711874 10343 raft_consensus.cc:482] T 2b5f64c69b4245de92fc32b1a711faa7 P 1522aa3f9fe54b1d95e66ff625693695 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:15:03.712137 10343 raft_consensus.cc:504] T 2b5f64c69b4245de92fc32b1a711faa7 P 1522aa3f9fe54b1d95e66ff625693695 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6da22627b28b4b19bef57184d3167576" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 46401 } } peers { permanent_uuid: "1522aa3f9fe54b1d95e66ff625693695" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 37533 } } peers { permanent_uuid: "f0dda022ce3140e08642813b191a3d19" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36913 } }
I20260806 21:15:03.713335 10343 leader_election.cc:288] T 2b5f64c69b4245de92fc32b1a711faa7 P 1522aa3f9fe54b1d95e66ff625693695 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6da22627b28b4b19bef57184d3167576 (127.16.178.3:46401), f0dda022ce3140e08642813b191a3d19 (127.16.178.1:36913)
I20260806 21:15:03.718559 9941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:03.713797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:42795 (local address 127.16.178.1:36913)
0806 21:15:03.713929 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:03.713933 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:03.714102 (+ 169us) server_negotiation.cc:407] Connection header received
0806 21:15:03.714207 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:03.714210 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:03.714276 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:03.714370 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:15:03.714580 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:03.715370 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:03.716997 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:03.717711 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:03.717821 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:03.718000 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:03.718276 (+ 276us) server_negotiation.cc:299] Negotiation successful
0806 21:15:03.718396 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 21:15:03.718559 10331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:03.713695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:49319 (local address 127.16.178.3:46401)
0806 21:15:03.713826 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:03.713830 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:03.713848 (+ 18us) server_negotiation.cc:407] Connection header received
0806 21:15:03.714011 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:03.714014 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:03.714058 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:03.714140 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:15:03.714380 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:03.715158 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:03.716997 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:03.717645 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:03.717808 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:03.717874 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:03.718276 (+ 402us) server_negotiation.cc:299] Negotiation successful
0806 21:15:03.718396 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 21:15:03.718554 10131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:03.713553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.3:46401 (local address 127.16.178.2:49319)
0806 21:15:03.713795 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:03.713811 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:03.713902 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:03.714161 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:03.714165 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:03.714179 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:15:03.714284 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:03.714289 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:03.715299 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:03.715302 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:03.716769 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:03.716787 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:03.717842 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:03.717847 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:03.718057 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:03.718113 (+ 56us) client_negotiation.cc:769] Sending connection context
0806 21:15:03.718210 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 21:15:03.718380 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":102}
I20260806 21:15:03.718564 10345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:03.713574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:36913 (local address 127.16.178.2:42795)
0806 21:15:03.714025 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:03.714036 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:03.714112 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:03.714387 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:03.714389 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:03.714404 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:15:03.714487 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:03.714491 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:03.715467 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:03.715470 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:03.716769 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:03.716787 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:03.717842 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:03.717847 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:03.718057 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:03.718113 (+ 56us) client_negotiation.cc:769] Sending connection context
0806 21:15:03.718210 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 21:15:03.718320 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"mutex_wait_us":124,"thread_start_us":138,"threads_started":1}
I20260806 21:15:03.719403 9856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b5f64c69b4245de92fc32b1a711faa7" candidate_uuid: "1522aa3f9fe54b1d95e66ff625693695" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0dda022ce3140e08642813b191a3d19" is_pre_election: true
I20260806 21:15:03.719513 10252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b5f64c69b4245de92fc32b1a711faa7" candidate_uuid: "1522aa3f9fe54b1d95e66ff625693695" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6da22627b28b4b19bef57184d3167576" is_pre_election: true
I20260806 21:15:03.719842 10252 raft_consensus.cc:2454] T 2b5f64c69b4245de92fc32b1a711faa7 P 6da22627b28b4b19bef57184d3167576 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1522aa3f9fe54b1d95e66ff625693695 in term 0.
I20260806 21:15:03.719843 9856 raft_consensus.cc:2454] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1522aa3f9fe54b1d95e66ff625693695 in term 0.
I20260806 21:15:03.720510 9964 leader_election.cc:302] T 2b5f64c69b4245de92fc32b1a711faa7 P 1522aa3f9fe54b1d95e66ff625693695 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1522aa3f9fe54b1d95e66ff625693695, 6da22627b28b4b19bef57184d3167576; no voters:
I20260806 21:15:03.720854 10343 raft_consensus.cc:2781] T 2b5f64c69b4245de92fc32b1a711faa7 P 1522aa3f9fe54b1d95e66ff625693695 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:15:03.720960 10343 raft_consensus.cc:482] T 2b5f64c69b4245de92fc32b1a711faa7 P 1522aa3f9fe54b1d95e66ff625693695 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:15:03.721025 10343 raft_consensus.cc:3037] T 2b5f64c69b4245de92fc32b1a711faa7 P 1522aa3f9fe54b1d95e66ff625693695 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:15:03.722216 10343 raft_consensus.cc:504] T 2b5f64c69b4245de92fc32b1a711faa7 P 1522aa3f9fe54b1d95e66ff625693695 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6da22627b28b4b19bef57184d3167576" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 46401 } } peers { permanent_uuid: "1522aa3f9fe54b1d95e66ff625693695" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 37533 } } peers { permanent_uuid: "f0dda022ce3140e08642813b191a3d19" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36913 } }
I20260806 21:15:03.722651 10343 leader_election.cc:288] T 2b5f64c69b4245de92fc32b1a711faa7 P 1522aa3f9fe54b1d95e66ff625693695 [CANDIDATE]: Term 1 election: Requested vote from peers 6da22627b28b4b19bef57184d3167576 (127.16.178.3:46401), f0dda022ce3140e08642813b191a3d19 (127.16.178.1:36913)
I20260806 21:15:03.723044 10252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b5f64c69b4245de92fc32b1a711faa7" candidate_uuid: "1522aa3f9fe54b1d95e66ff625693695" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6da22627b28b4b19bef57184d3167576"
I20260806 21:15:03.723112 9856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b5f64c69b4245de92fc32b1a711faa7" candidate_uuid: "1522aa3f9fe54b1d95e66ff625693695" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0dda022ce3140e08642813b191a3d19"
I20260806 21:15:03.723258 9856 raft_consensus.cc:3037] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:15:03.723214 10252 raft_consensus.cc:3037] T 2b5f64c69b4245de92fc32b1a711faa7 P 6da22627b28b4b19bef57184d3167576 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:15:03.724504 9856 raft_consensus.cc:2454] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1522aa3f9fe54b1d95e66ff625693695 in term 1.
I20260806 21:15:03.724516 10252 raft_consensus.cc:2454] T 2b5f64c69b4245de92fc32b1a711faa7 P 6da22627b28b4b19bef57184d3167576 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1522aa3f9fe54b1d95e66ff625693695 in term 1.
I20260806 21:15:03.724923 9964 leader_election.cc:302] T 2b5f64c69b4245de92fc32b1a711faa7 P 1522aa3f9fe54b1d95e66ff625693695 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1522aa3f9fe54b1d95e66ff625693695, 6da22627b28b4b19bef57184d3167576; no voters:
I20260806 21:15:03.725148 10343 raft_consensus.cc:2781] T 2b5f64c69b4245de92fc32b1a711faa7 P 1522aa3f9fe54b1d95e66ff625693695 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:15:03.725395 10343 raft_consensus.cc:686] T 2b5f64c69b4245de92fc32b1a711faa7 P 1522aa3f9fe54b1d95e66ff625693695 [term 1 LEADER]: Becoming Leader. State: Replica: 1522aa3f9fe54b1d95e66ff625693695, State: Running, Role: LEADER
I20260806 21:15:03.725809 10343 consensus_queue.cc:237] T 2b5f64c69b4245de92fc32b1a711faa7 P 1522aa3f9fe54b1d95e66ff625693695 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6da22627b28b4b19bef57184d3167576" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 46401 } } peers { permanent_uuid: "1522aa3f9fe54b1d95e66ff625693695" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 37533 } } peers { permanent_uuid: "f0dda022ce3140e08642813b191a3d19" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36913 } }
I20260806 21:15:03.729971 9639 catalog_manager.cc:5495] T 2b5f64c69b4245de92fc32b1a711faa7 P 1522aa3f9fe54b1d95e66ff625693695 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1522aa3f9fe54b1d95e66ff625693695 (127.16.178.2). New cstate: current_term: 1 leader_uuid: "1522aa3f9fe54b1d95e66ff625693695" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6da22627b28b4b19bef57184d3167576" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 46401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1522aa3f9fe54b1d95e66ff625693695" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 37533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f0dda022ce3140e08642813b191a3d19" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36913 } health_report { overall_health: UNKNOWN } } }
I20260806 21:15:03.745890 10334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:03.742202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40682 (local address 127.16.178.2:37533)
0806 21:15:03.742320 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:03.742324 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:03.742338 (+ 14us) server_negotiation.cc:407] Connection header received
0806 21:15:03.742556 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:03.742559 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:03.742628 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:03.742742 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:15:03.742989 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:03.743490 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:03.744431 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:03.744660 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:03.744758 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:03.745041 (+ 283us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:15:03.745514 (+ 473us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:15:03.745611 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:03.745678 (+ 67us) server_negotiation.cc:299] Negotiation successful
0806 21:15:03.745739 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 21:15:03.752831 10252 raft_consensus.cc:1261] T 2b5f64c69b4245de92fc32b1a711faa7 P 6da22627b28b4b19bef57184d3167576 [term 1 FOLLOWER]: Refusing update from remote peer 1522aa3f9fe54b1d95e66ff625693695: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:15:03.752852 9856 raft_consensus.cc:1261] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19 [term 1 FOLLOWER]: Refusing update from remote peer 1522aa3f9fe54b1d95e66ff625693695: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:15:03.753655 10346 consensus_queue.cc:1035] T 2b5f64c69b4245de92fc32b1a711faa7 P 1522aa3f9fe54b1d95e66ff625693695 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6da22627b28b4b19bef57184d3167576" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 46401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:15:03.753852 10343 consensus_queue.cc:1035] T 2b5f64c69b4245de92fc32b1a711faa7 P 1522aa3f9fe54b1d95e66ff625693695 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0dda022ce3140e08642813b191a3d19" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:15:03.760828 10350 mvcc.cc:205] Tried to move back new op lower bound from 7315664501763350528 to 7315664501664301056. Current Snapshot: MvccSnapshot[applied={T|T < 7315664501763350528}]
I20260806 21:15:03.762833 10351 mvcc.cc:205] Tried to move back new op lower bound from 7315664501763350528 to 7315664501664301056. Current Snapshot: MvccSnapshot[applied={T|T < 7315664501763350528}]
I20260806 21:15:03.771600 9639 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40274:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:15:03.771824 9639 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40274:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:15:03.774612 9639 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7: Sending DeleteTablet for 3 replicas of tablet 2b5f64c69b4245de92fc32b1a711faa7
I20260806 21:15:03.775566 10032 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b5f64c69b4245de92fc32b1a711faa7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:15:03 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:40678
I20260806 21:15:03.775584 10220 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b5f64c69b4245de92fc32b1a711faa7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:15:03 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:53226
I20260806 21:15:03.776146 9825 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b5f64c69b4245de92fc32b1a711faa7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:15:03 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:59882
I20260806 21:15:03.776248 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 9699
I20260806 21:15:03.776398 10362 tablet_replica.cc:330] T 2b5f64c69b4245de92fc32b1a711faa7 P 6da22627b28b4b19bef57184d3167576: stopping tablet replica
I20260806 21:15:03.776587 10361 tablet_replica.cc:330] T 2b5f64c69b4245de92fc32b1a711faa7 P 1522aa3f9fe54b1d95e66ff625693695: stopping tablet replica
I20260806 21:15:03.783136 10362 raft_consensus.cc:2229] T 2b5f64c69b4245de92fc32b1a711faa7 P 6da22627b28b4b19bef57184d3167576 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:15:03.783635 10361 raft_consensus.cc:2229] T 2b5f64c69b4245de92fc32b1a711faa7 P 1522aa3f9fe54b1d95e66ff625693695 [term 1 LEADER]: Raft consensus shutting down.
I20260806 21:15:03.784273 10361 raft_consensus.cc:2258] T 2b5f64c69b4245de92fc32b1a711faa7 P 1522aa3f9fe54b1d95e66ff625693695 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:15:03.784691 10362 raft_consensus.cc:2258] T 2b5f64c69b4245de92fc32b1a711faa7 P 6da22627b28b4b19bef57184d3167576 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:15:03.787070 10362 ts_tablet_manager.cc:1905] T 2b5f64c69b4245de92fc32b1a711faa7 P 6da22627b28b4b19bef57184d3167576: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:15:03.790045 10362 ts_tablet_manager.cc:1918] T 2b5f64c69b4245de92fc32b1a711faa7 P 6da22627b28b4b19bef57184d3167576: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:15:03.790165 10362 log.cc:1196] T 2b5f64c69b4245de92fc32b1a711faa7 P 6da22627b28b4b19bef57184d3167576: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-2/wal/wals/2b5f64c69b4245de92fc32b1a711faa7
I20260806 21:15:03.790517 10362 ts_tablet_manager.cc:1939] T 2b5f64c69b4245de92fc32b1a711faa7 P 6da22627b28b4b19bef57184d3167576: Deleting consensus metadata
I20260806 21:15:03.792017 9620 catalog_manager.cc:4841] TS 6da22627b28b4b19bef57184d3167576 (127.16.178.3:46401): tablet 2b5f64c69b4245de92fc32b1a711faa7 (table test-table [id=9b3180249a0145abbed68c33c935aa45]) successfully deleted
I20260806 21:15:03.793287 10361 ts_tablet_manager.cc:1905] T 2b5f64c69b4245de92fc32b1a711faa7 P 1522aa3f9fe54b1d95e66ff625693695: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:15:03.795713 10361 ts_tablet_manager.cc:1918] T 2b5f64c69b4245de92fc32b1a711faa7 P 1522aa3f9fe54b1d95e66ff625693695: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:15:03.795815 10361 log.cc:1196] T 2b5f64c69b4245de92fc32b1a711faa7 P 1522aa3f9fe54b1d95e66ff625693695: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-1/wal/wals/2b5f64c69b4245de92fc32b1a711faa7
I20260806 21:15:03.796131 10361 ts_tablet_manager.cc:1939] T 2b5f64c69b4245de92fc32b1a711faa7 P 1522aa3f9fe54b1d95e66ff625693695: Deleting consensus metadata
I20260806 21:15:03.797377 9621 catalog_manager.cc:4841] TS 1522aa3f9fe54b1d95e66ff625693695 (127.16.178.2:37533): tablet 2b5f64c69b4245de92fc32b1a711faa7 (table test-table [id=9b3180249a0145abbed68c33c935aa45]) successfully deleted
W20260806 21:15:03.798405 9619 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.178.1:36913 (error 108)
I20260806 21:15:03.799100 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 9938
I20260806 21:15:03.802215 10333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:03.799286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:36913 (local address 127.0.0.1:59884)
0806 21:15:03.802015 (+ 2729us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:03.802113 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.1:36913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2472}
W20260806 21:15:03.802554 9619 catalog_manager.cc:4584] TS f0dda022ce3140e08642813b191a3d19 (127.16.178.1:36913): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2b5f64c69b4245de92fc32b1a711faa7: Network error: Client connection negotiation failed: client connection to 127.16.178.1:36913: connect: Connection refused (error 111)
I20260806 21:15:03.812146 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 10134
I20260806 21:15:03.839936 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 9586
I20260806 21:15:23.857617 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:37115
--webserver_interface=127.16.178.62
--webserver_port=44117
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:45599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:37115
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:15:23.981129 10364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:15:23.981438 10364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:15:23.981487 10364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:15:23.985734 10364 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 21:15:23.985800 10364 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 21:15:23.985816 10364 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 21:15:23.985828 10364 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:15:23.985854 10364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:15:23.985888 10364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:15:23.985916 10364 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:15:23.985944 10364 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:15:23.991725 10364 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:45599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:37115
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:37115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.178.62
--webserver_port=44117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:15:23.993021 10364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:15:23.994122 10364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:15:24.003990 10374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:24.004015 10372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:24.004123 10364 server_base.cc:1034] running on GCE node
W20260806 21:15:24.003990 10371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:24.005059 10364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:15:24.006318 10364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:15:24.007530 10364 hybrid_clock.cc:648] HybridClock initialized: now 1786050924007494 us; error 50 us; skew 500 ppm
Aug 06 21:15:24 dist-test-slave-lgr5 krb5kdc[8747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050924, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:15:24.017195 10364 init.cc:377] Logged in from keytab as kudu/127.16.178.62@KRBTEST.COM (short username kudu)
I20260806 21:15:24.018373 10364 webserver.cc:460] Webserver started at http://127.16.178.62:44117/ using document root <none> and password file <none>
I20260806 21:15:24.019043 10364 fs_manager.cc:359] Metadata directory not provided
I20260806 21:15:24.019124 10364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:15:24.023517 10364 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:15:24.026087 10383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:24.027392 10364 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:15:24.027534 10364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "82fdec997fa9475aa81fc138551e36e7"
format_stamp: "Formatted at 2026-08-06 21:14:41 on dist-test-slave-lgr5"
server_key: "e1354ebc86ec56fef3422989f77ade89"
server_key_iv: "437b13bb76db6a206a5f5e11e4f0132e"
server_key_version: "encryptionkey@0"
I20260806 21:15:24.028199 10364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:15:24.046830 10364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:15:24.050105 10364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:15:24.050309 10364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:15:24.057668 10364 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.62:37115
I20260806 21:15:24.057964 10459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.62:37115 every 8 connection(s)
I20260806 21:15:24.058883 10364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:15:24.063800 10460 sys_catalog.cc:263] Verifying existing consensus state
I20260806 21:15:24.065716 10460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7: Bootstrap starting.
I20260806 21:15:24.066298 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 10364
I20260806 21:15:24.066838 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.1:36913
--local_ip_for_outbound_sockets=127.16.178.1
--tserver_master_addrs=127.16.178.62:37115
--webserver_port=40463
--webserver_interface=127.16.178.1
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab
--principal=kudu/127.16.178.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:45599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 21:15:24.085331 10460 log.cc:824] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7: Log is configured to *not* fsync() on all Append() calls
I20260806 21:15:24.097246 10460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 21:15:24.097666 10460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7: Bootstrap complete.
I20260806 21:15:24.101828 10460 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fdec997fa9475aa81fc138551e36e7" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 37115 } }
I20260806 21:15:24.102326 10460 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82fdec997fa9475aa81fc138551e36e7, State: Initialized, Role: FOLLOWER
I20260806 21:15:24.102867 10460 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fdec997fa9475aa81fc138551e36e7" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 37115 } }
I20260806 21:15:24.102996 10460 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:15:24.103062 10460 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:15:24.103163 10460 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [term 2 FOLLOWER]: Advancing to term 3
I20260806 21:15:24.105178 10460 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fdec997fa9475aa81fc138551e36e7" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 37115 } }
I20260806 21:15:24.105508 10460 leader_election.cc:302] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82fdec997fa9475aa81fc138551e36e7; no voters:
I20260806 21:15:24.105808 10460 leader_election.cc:288] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 21:15:24.105952 10466 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 21:15:24.106215 10466 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [term 3 LEADER]: Becoming Leader. State: Replica: 82fdec997fa9475aa81fc138551e36e7, State: Running, Role: LEADER
I20260806 21:15:24.106554 10466 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fdec997fa9475aa81fc138551e36e7" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 37115 } }
I20260806 21:15:24.106971 10460 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:15:24.107604 10466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82fdec997fa9475aa81fc138551e36e7. Latest consensus state: current_term: 3 leader_uuid: "82fdec997fa9475aa81fc138551e36e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fdec997fa9475aa81fc138551e36e7" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 37115 } } }
I20260806 21:15:24.107724 10466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [sys.catalog]: This master's current role is: LEADER
I20260806 21:15:24.108369 10467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "82fdec997fa9475aa81fc138551e36e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fdec997fa9475aa81fc138551e36e7" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 37115 } } }
I20260806 21:15:24.108484 10467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7 [sys.catalog]: This master's current role is: LEADER
I20260806 21:15:24.109061 10474 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:15:24.114535 10474 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:15:24.115290 10474 catalog_manager.cc:1259] Loaded cluster ID: 0ac10d4413944f2293d8ec7cb6b3a3d3
I20260806 21:15:24.115357 10474 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:15:24.118641 10474 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:15:24.119459 10474 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7: Loaded TSK: 0
I20260806 21:15:24.119550 10474 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 82fdec997fa9475aa81fc138551e36e7: Loaded TSK: 1
F20260806 21:15:24.120333 10474 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f8d13df99ad google::LogMessage::Fail() at ??:0
@ 0x7f8d13dfc88a google::LogMessage::SendToLog() at ??:0
@ 0x7f8d13df94de google::LogMessage::Flush() at ??:0
@ 0x7f8d13dfaa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f8d16abd18d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f8d16ab5f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f8d1e75e7bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f8d1e73899c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f8d1e739178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f8d1e77adf8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f8d1e79982b std::function<>::operator()() at ??:0
@ 0x7f8d1e738b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f8d1e739f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f8d1e736579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f8d1e77a640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f8d1604b3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f8d1604bc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f8d1604d3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f8d1603c512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f8d176806db start_thread at ??:0
@ 0x7f8d128f761f clone at ??:0
W20260806 21:15:24.213829 10463 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:15:24.214118 10463 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:15:24.214164 10463 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:15:24.218681 10463 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:15:24.218740 10463 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:15:24.218801 10463 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.1
I20260806 21:15:24.224783 10463 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:45599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab
--principal=kudu/127.16.178.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.1:36913
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.178.1
--webserver_port=40463
--tserver_master_addrs=127.16.178.62:37115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10463
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.1
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:15:24.225970 10463 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:15:24.227054 10463 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:15:24.238173 10528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:24.238158 10529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:24.238430 10532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:24.238561 10463 server_base.cc:1034] running on GCE node
I20260806 21:15:24.239089 10463 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:15:24.239923 10463 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:15:24.241132 10463 hybrid_clock.cc:648] HybridClock initialized: now 1786050924241106 us; error 39 us; skew 500 ppm
Aug 06 21:15:24 dist-test-slave-lgr5 krb5kdc[8747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050924, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:15:24.244654 10463 init.cc:377] Logged in from keytab as kudu/127.16.178.1@KRBTEST.COM (short username kudu)
I20260806 21:15:24.245662 10463 webserver.cc:460] Webserver started at http://127.16.178.1:40463/ using document root <none> and password file <none>
I20260806 21:15:24.246248 10463 fs_manager.cc:359] Metadata directory not provided
I20260806 21:15:24.246311 10463 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:15:24.250656 10463 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 21:15:24.253386 10540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:24.255120 10463 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:15:24.255272 10463 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-0/wal
uuid: "f0dda022ce3140e08642813b191a3d19"
format_stamp: "Formatted at 2026-08-06 21:14:42 on dist-test-slave-lgr5"
server_key: "3ef4935787c7a2b933e6df7f4350afb1"
server_key_iv: "a8e0caa24230334a4dbd9d7a9fbfa49a"
server_key_version: "encryptionkey@0"
I20260806 21:15:24.255897 10463 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:15:24.275389 10463 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:15:24.278838 10463 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:15:24.279036 10463 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:15:24.279733 10463 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:15:24.280980 10547 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 21:15:24.287035 10463 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 21:15:24.287137 10463 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260806 21:15:24.287215 10463 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 21:15:24.289106 10463 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 21:15:24.289162 10463 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260806 21:15:24.289551 10547 tablet_bootstrap.cc:492] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19: Bootstrap starting.
I20260806 21:15:24.300202 10463 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.1:36913
I20260806 21:15:24.300552 10701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.1:36913 every 8 connection(s)
I20260806 21:15:24.301604 10463 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
I20260806 21:15:24.305497 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 10463
I20260806 21:15:24.306080 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.2:37533
--local_ip_for_outbound_sockets=127.16.178.2
--tserver_master_addrs=127.16.178.62:37115
--webserver_port=44737
--webserver_interface=127.16.178.2
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab
--principal=kudu/127.16.178.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:45599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 21:15:24.316758 10547 log.cc:824] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19: Log is configured to *not* fsync() on all Append() calls
I20260806 21:15:24.324688 10547 tablet_bootstrap.cc:492] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 21:15:24.325296 10547 tablet_bootstrap.cc:492] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19: Bootstrap complete.
I20260806 21:15:24.326602 10547 ts_tablet_manager.cc:1397] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19: Time spent bootstrapping tablet: real 0.037s user 0.025s sys 0.008s
I20260806 21:15:24.330163 10547 raft_consensus.cc:348] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6da22627b28b4b19bef57184d3167576" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 46401 } } peers { permanent_uuid: "1522aa3f9fe54b1d95e66ff625693695" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 37533 } } peers { permanent_uuid: "f0dda022ce3140e08642813b191a3d19" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36913 } }
I20260806 21:15:24.330775 10547 raft_consensus.cc:729] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0dda022ce3140e08642813b191a3d19, State: Initialized, Role: FOLLOWER
I20260806 21:15:24.331341 10547 consensus_queue.cc:260] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6da22627b28b4b19bef57184d3167576" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 46401 } } peers { permanent_uuid: "1522aa3f9fe54b1d95e66ff625693695" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 37533 } } peers { permanent_uuid: "f0dda022ce3140e08642813b191a3d19" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36913 } }
I20260806 21:15:24.332521 10547 ts_tablet_manager.cc:1428] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260806 21:15:24.410893 10704 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:15:24.303862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.1:52497)
0806 21:15:24.304489 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:24.304560 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:24.305748 (+ 1188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:24.410054 (+104306us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":195,"threads_started":1}
W20260806 21:15:24.411550 10702 heartbeater.cc:644] Failed to heartbeat to 127.16.178.62:37115 (0 consecutive failures): Network error: Failed to ping master at 127.16.178.62:37115: Client connection negotiation failed: client connection to 127.16.178.62:37115: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 21:15:24.412472 10704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:24.412157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.1:48559)
0806 21:15:24.412337 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:24.412409 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260806 21:15:24.413148 10704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:24.412933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.1:46467)
0806 21:15:24.413037 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:24.413095 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260806 21:15:24.413318 10702 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 21:15:24.458168 10705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:15:24.458504 10705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:15:24.458564 10705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:15:24.463017 10705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:15:24.463116 10705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:15:24.463192 10705 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.2
I20260806 21:15:24.468918 10705 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:45599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab
--principal=kudu/127.16.178.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.2:37533
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.178.2
--webserver_port=44737
--tserver_master_addrs=127.16.178.62:37115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.2
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:15:24.470180 10705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:15:24.471318 10705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:15:24.481737 10715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:24.481767 10717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:24.481755 10714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:24.482048 10705 server_base.cc:1034] running on GCE node
I20260806 21:15:24.482635 10705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:15:24.483376 10705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:15:24.484571 10705 hybrid_clock.cc:648] HybridClock initialized: now 1786050924484538 us; error 42 us; skew 500 ppm
Aug 06 21:15:24 dist-test-slave-lgr5 krb5kdc[8747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050924, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:15:24.487561 10705 init.cc:377] Logged in from keytab as kudu/127.16.178.2@KRBTEST.COM (short username kudu)
I20260806 21:15:24.488658 10705 webserver.cc:460] Webserver started at http://127.16.178.2:44737/ using document root <none> and password file <none>
I20260806 21:15:24.489250 10705 fs_manager.cc:359] Metadata directory not provided
I20260806 21:15:24.489336 10705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:15:24.493544 10705 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 21:15:24.496006 10727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:24.497407 10705 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:15:24.497538 10705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-1/wal
uuid: "1522aa3f9fe54b1d95e66ff625693695"
format_stamp: "Formatted at 2026-08-06 21:14:42 on dist-test-slave-lgr5"
server_key: "3733ff686be7eb1cfb4140b162b931f4"
server_key_iv: "84e3ed641f1ef6253e1969d8d22d01b0"
server_key_version: "encryptionkey@0"
I20260806 21:15:24.498121 10705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:15:24.516950 10705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:15:24.520288 10705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:15:24.520531 10705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:15:24.521195 10705 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:15:24.522136 10705 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:15:24.522188 10705 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:24.522248 10705 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:15:24.522275 10705 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:24.531368 10705 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.2:37533
I20260806 21:15:24.531420 10911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.2:37533 every 8 connection(s)
I20260806 21:15:24.532485 10705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
I20260806 21:15:24.536458 10914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:24.534697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.2:36995)
0806 21:15:24.535204 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:24.535675 (+ 471us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":109,"threads_started":1}
W20260806 21:15:24.536939 10912 heartbeater.cc:644] Failed to heartbeat to 127.16.178.62:37115 (0 consecutive failures): Network error: Failed to ping master at 127.16.178.62:37115: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
I20260806 21:15:24.537549 10914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:24.537336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.2:59523)
0806 21:15:24.537441 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:24.537491 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 21:15:24.538115 10914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:24.537955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.2:44247)
0806 21:15:24.538034 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:24.538072 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 21:15:24.538282 10912 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 21:15:24.540285 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 10705
I20260806 21:15:24.540688 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.3:46401
--local_ip_for_outbound_sockets=127.16.178.3
--tserver_master_addrs=127.16.178.62:37115
--webserver_port=44075
--webserver_interface=127.16.178.3
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab
--principal=kudu/127.16.178.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:45599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:15:24.660669 10916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:15:24.660970 10916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:15:24.661031 10916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:15:24.665155 10916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:15:24.665235 10916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:15:24.665297 10916 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.3
I20260806 21:15:24.670837 10916 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:45599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab
--principal=kudu/127.16.178.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.3:46401
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.178.3
--webserver_port=44075
--tserver_master_addrs=127.16.178.62:37115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.3
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:15:24.672056 10916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:15:24.673197 10916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:15:24.683974 10927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:24.683967 10924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:24.684137 10916 server_base.cc:1034] running on GCE node
W20260806 21:15:24.683967 10930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:24.684962 10916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:15:24.685786 10916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:15:24.686983 10916 hybrid_clock.cc:648] HybridClock initialized: now 1786050924686950 us; error 51 us; skew 500 ppm
Aug 06 21:15:24 dist-test-slave-lgr5 krb5kdc[8747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050924, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:15:24.690168 10916 init.cc:377] Logged in from keytab as kudu/127.16.178.3@KRBTEST.COM (short username kudu)
I20260806 21:15:24.691262 10916 webserver.cc:460] Webserver started at http://127.16.178.3:44075/ using document root <none> and password file <none>
I20260806 21:15:24.691907 10916 fs_manager.cc:359] Metadata directory not provided
I20260806 21:15:24.691992 10916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:15:24.696336 10916 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:15:24.698585 10941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:24.699928 10916 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:15:24.700074 10916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-2/wal
uuid: "6da22627b28b4b19bef57184d3167576"
format_stamp: "Formatted at 2026-08-06 21:14:42 on dist-test-slave-lgr5"
server_key: "35923fcda49e926fa5e11018e8e2146a"
server_key_iv: "6b2e9dc722f6d970b1a5caa219436a04"
server_key_version: "encryptionkey@0"
I20260806 21:15:24.700686 10916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:15:24.718061 10916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:15:24.721479 10916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:15:24.721704 10916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:15:24.722373 10916 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:15:24.723299 10916 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:15:24.723351 10916 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:24.723409 10916 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:15:24.723439 10916 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:24.732753 10916 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.3:46401
I20260806 21:15:24.732825 11103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.3:46401 every 8 connection(s)
I20260806 21:15:24.733841 10916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
I20260806 21:15:24.737802 11107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:24.736036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.3:54793)
0806 21:15:24.736621 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:24.737075 (+ 454us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":117,"threads_started":1}
W20260806 21:15:24.738250 11105 heartbeater.cc:644] Failed to heartbeat to 127.16.178.62:37115 (0 consecutive failures): Network error: Failed to ping master at 127.16.178.62:37115: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
I20260806 21:15:24.738903 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 10916
I20260806 21:15:24.738973 11107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:24.738682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.3:50225)
0806 21:15:24.738842 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:24.738906 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 21:15:24.739568 11107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:24.739371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.3:47251)
0806 21:15:24.739473 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:24.739518 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260806 21:15:24.739778 11105 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 21:15:25.414479 11111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:25.413970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.1:44719)
0806 21:15:25.414285 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:25.414357 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":85,"threads_started":1}
I20260806 21:15:25.539389 11112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:25.538934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.2:55461)
0806 21:15:25.539203 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:25.539265 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":90,"threads_started":1}
I20260806 21:15:25.590493 11114 raft_consensus.cc:482] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:15:25.590741 11114 raft_consensus.cc:504] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6da22627b28b4b19bef57184d3167576" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 46401 } } peers { permanent_uuid: "1522aa3f9fe54b1d95e66ff625693695" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 37533 } } peers { permanent_uuid: "f0dda022ce3140e08642813b191a3d19" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36913 } }
I20260806 21:15:25.592125 11114 leader_election.cc:288] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6da22627b28b4b19bef57184d3167576 (127.16.178.3:46401), 1522aa3f9fe54b1d95e66ff625693695 (127.16.178.2:37533)
Aug 06 21:15:25 dist-test-slave-lgr5 krb5kdc[8747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050924, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for kudu/127.16.178.2@KRBTEST.COM
Aug 06 21:15:25 dist-test-slave-lgr5 krb5kdc[8747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050924, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for kudu/127.16.178.3@KRBTEST.COM
I20260806 21:15:25.612634 11116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:25.592593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:36689 (local address 127.16.178.2:37533)
0806 21:15:25.593135 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:25.593153 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:25.593187 (+ 34us) server_negotiation.cc:407] Connection header received
0806 21:15:25.594197 (+ 1010us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:25.594208 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:25.594725 (+ 517us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:25.595121 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:15:25.597195 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:25.598870 (+ 1675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:25.601055 (+ 2185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:25.601930 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:25.602148 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:25.606408 (+ 4260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:15:25.606416 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:15:25.606436 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:15:25.606481 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:15:25.608912 (+ 2431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:25.609835 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:25.609844 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:25.609850 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:25.610049 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:25.610511 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:25.610515 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:25.610517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:25.611126 (+ 609us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:15:25.611266 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:25.612075 (+ 809us) server_negotiation.cc:299] Negotiation successful
0806 21:15:25.612434 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":123,"threads_started":1}
I20260806 21:15:25.612634 11117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:25.592693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:41097 (local address 127.16.178.3:46401)
0806 21:15:25.593052 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:25.593070 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:25.593440 (+ 370us) server_negotiation.cc:407] Connection header received
0806 21:15:25.594322 (+ 882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:25.594330 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:25.594724 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:25.595121 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:15:25.597195 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:25.598880 (+ 1685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:25.601054 (+ 2174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:25.601930 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:25.602135 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:25.606408 (+ 4273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:15:25.606416 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:15:25.606437 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:15:25.606482 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:15:25.608912 (+ 2430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:25.609835 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:25.609844 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:25.609850 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:25.610049 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:25.610511 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:25.610515 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:25.610517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:25.611126 (+ 609us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:15:25.611266 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:25.612075 (+ 809us) server_negotiation.cc:299] Negotiation successful
0806 21:15:25.612452 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":87,"threads_started":1}
I20260806 21:15:25.612627 11118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:25.592407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.3:46401 (local address 127.16.178.1:41097)
0806 21:15:25.593318 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:25.593332 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:25.593451 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:25.595263 (+ 1812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:25.595270 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:25.595929 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:15:25.596913 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:25.596942 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:25.599097 (+ 2155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:25.599101 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:25.600874 (+ 1773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:25.600890 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:25.602373 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:25.602378 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:25.602585 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:25.603445 (+ 860us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:15:25.603473 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:15:25.606090 (+ 2617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:15:25.609096 (+ 3006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:15:25.609105 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:15:25.609124 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:15:25.609660 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:15:25.610155 (+ 495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:15:25.610158 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:15:25.610160 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:15:25.610404 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:15:25.611270 (+ 866us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:15:25.611279 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:15:25.611657 (+ 378us) client_negotiation.cc:769] Sending connection context
0806 21:15:25.611906 (+ 249us) client_negotiation.cc:240] Negotiation successful
0806 21:15:25.612413 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":829,"mutex_wait_us":880,"spinlock_wait_cycles":27008,"thread_start_us":93,"threads_started":1}
I20260806 21:15:25.612722 11111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:25.592407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.2:37533 (local address 127.16.178.1:36689)
0806 21:15:25.592646 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:25.592664 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:25.592783 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:25.595263 (+ 2480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:25.595270 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:25.595929 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:15:25.596913 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:25.596942 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:25.599097 (+ 2155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:25.599101 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:25.600875 (+ 1774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:25.600890 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:25.602274 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:25.602280 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:25.602607 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:25.603613 (+ 1006us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:15:25.603628 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:15:25.606090 (+ 2462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:15:25.609096 (+ 3006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:15:25.609105 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:15:25.609124 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:15:25.609660 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:15:25.610155 (+ 495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:15:25.610158 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:15:25.610160 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:15:25.610404 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:15:25.611270 (+ 866us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:15:25.611279 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:15:25.611657 (+ 378us) client_negotiation.cc:769] Sending connection context
0806 21:15:25.611906 (+ 249us) client_negotiation.cc:240] Negotiation successful
0806 21:15:25.612559 (+ 653us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":637,"spinlock_wait_cycles":92544}
W20260806 21:15:25.615381 10544 leader_election.cc:341] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1522aa3f9fe54b1d95e66ff625693695 (127.16.178.2:37533): Not found: Tablet not found: 2b5f64c69b4245de92fc32b1a711faa7
W20260806 21:15:25.615650 10543 leader_election.cc:341] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6da22627b28b4b19bef57184d3167576 (127.16.178.3:46401): Not found: Tablet not found: 2b5f64c69b4245de92fc32b1a711faa7
I20260806 21:15:25.615743 10543 leader_election.cc:302] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f0dda022ce3140e08642813b191a3d19; no voters: 1522aa3f9fe54b1d95e66ff625693695, 6da22627b28b4b19bef57184d3167576
I20260806 21:15:25.616046 11114 raft_consensus.cc:2726] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 21:15:25.740869 11119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:25.740380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.3:47187)
0806 21:15:25.740680 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:25.740747 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":105,"threads_started":1}
I20260806 21:15:26.416146 11120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:26.415530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.1:34647)
0806 21:15:26.415899 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:26.415981 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":91,"threads_started":1}
I20260806 21:15:26.540935 11121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:26.540412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.2:43829)
0806 21:15:26.540733 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:26.540809 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":99,"threads_started":1}
I20260806 21:15:26.742120 11122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:26.741652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.3:36297)
0806 21:15:26.741932 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:26.741995 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":82,"threads_started":1}
I20260806 21:15:27.172230 11123 raft_consensus.cc:482] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:15:27.172385 11123 raft_consensus.cc:504] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6da22627b28b4b19bef57184d3167576" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 46401 } } peers { permanent_uuid: "1522aa3f9fe54b1d95e66ff625693695" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 37533 } } peers { permanent_uuid: "f0dda022ce3140e08642813b191a3d19" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36913 } }
I20260806 21:15:27.172912 11123 leader_election.cc:288] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6da22627b28b4b19bef57184d3167576 (127.16.178.3:46401), 1522aa3f9fe54b1d95e66ff625693695 (127.16.178.2:37533)
W20260806 21:15:27.173825 10544 leader_election.cc:341] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1522aa3f9fe54b1d95e66ff625693695 (127.16.178.2:37533): Not found: Tablet not found: 2b5f64c69b4245de92fc32b1a711faa7
W20260806 21:15:27.173950 10543 leader_election.cc:341] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6da22627b28b4b19bef57184d3167576 (127.16.178.3:46401): Not found: Tablet not found: 2b5f64c69b4245de92fc32b1a711faa7
I20260806 21:15:27.174050 10543 leader_election.cc:302] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f0dda022ce3140e08642813b191a3d19; no voters: 1522aa3f9fe54b1d95e66ff625693695, 6da22627b28b4b19bef57184d3167576
I20260806 21:15:27.174209 11123 raft_consensus.cc:2726] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 21:15:27.417716 11126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:27.417222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.1:48213)
0806 21:15:27.417535 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:27.417606 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":91,"threads_started":1}
I20260806 21:15:27.542184 11127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:27.541668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.2:34087)
0806 21:15:27.541990 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:27.542057 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":86,"threads_started":1}
I20260806 21:15:27.743606 11130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:27.743110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.3:52717)
0806 21:15:27.743431 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:27.743500 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":106,"threads_started":1}
I20260806 21:15:28.419178 11131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:28.418684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.1:38001)
0806 21:15:28.418984 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:28.419062 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":88,"threads_started":1}
I20260806 21:15:28.543464 11132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:28.542940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.2:49105)
0806 21:15:28.543253 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:28.543317 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":85,"threads_started":1}
I20260806 21:15:28.745023 11133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:28.744503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.3:33941)
0806 21:15:28.744823 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:28.744893 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":99,"threads_started":1}
I20260806 21:15:29.420931 11134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:29.420395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.1:49203)
0806 21:15:29.420765 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:29.420831 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":86,"threads_started":1}
I20260806 21:15:29.544822 11136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:29.544369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.2:59025)
0806 21:15:29.544644 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:29.544708 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":82,"threads_started":1}
I20260806 21:15:29.746538 11139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:29.746004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.3:48611)
0806 21:15:29.746333 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:29.746403 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":93,"threads_started":1}
I20260806 21:15:30.196926 11140 raft_consensus.cc:482] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:15:30.197069 11140 raft_consensus.cc:504] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6da22627b28b4b19bef57184d3167576" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 46401 } } peers { permanent_uuid: "1522aa3f9fe54b1d95e66ff625693695" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 37533 } } peers { permanent_uuid: "f0dda022ce3140e08642813b191a3d19" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36913 } }
I20260806 21:15:30.197551 11140 leader_election.cc:288] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6da22627b28b4b19bef57184d3167576 (127.16.178.3:46401), 1522aa3f9fe54b1d95e66ff625693695 (127.16.178.2:37533)
W20260806 21:15:30.198305 10544 leader_election.cc:341] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1522aa3f9fe54b1d95e66ff625693695 (127.16.178.2:37533): Not found: Tablet not found: 2b5f64c69b4245de92fc32b1a711faa7
W20260806 21:15:30.198421 10543 leader_election.cc:341] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6da22627b28b4b19bef57184d3167576 (127.16.178.3:46401): Not found: Tablet not found: 2b5f64c69b4245de92fc32b1a711faa7
I20260806 21:15:30.198503 10543 leader_election.cc:302] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f0dda022ce3140e08642813b191a3d19; no voters: 1522aa3f9fe54b1d95e66ff625693695, 6da22627b28b4b19bef57184d3167576
I20260806 21:15:30.198647 11140 raft_consensus.cc:2726] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 21:15:30.422389 11141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:30.421911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.1:58961)
0806 21:15:30.422213 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:30.422280 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":82,"threads_started":1}
I20260806 21:15:30.546200 11144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:30.545697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.2:58757)
0806 21:15:30.545979 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:30.546088 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":84,"threads_started":1}
I20260806 21:15:30.748059 11145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:30.747571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.3:48851)
0806 21:15:30.747880 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:30.747955 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":81,"threads_started":1}
I20260806 21:15:31.423887 11146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:31.423345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.1:41571)
0806 21:15:31.423697 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:31.423772 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":87,"threads_started":1}
I20260806 21:15:31.547735 11147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:31.547150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.2:54123)
0806 21:15:31.547555 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:31.547627 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":92,"threads_started":1}
I20260806 21:15:31.749670 11148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:31.749105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.3:47029)
0806 21:15:31.749487 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:31.749557 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":102,"threads_started":1}
I20260806 21:15:32.425460 11150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:32.424863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.1:48573)
0806 21:15:32.425219 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:32.425295 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":88,"threads_started":1}
I20260806 21:15:32.549058 11151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:32.548629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.2:36119)
0806 21:15:32.548884 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:32.548943 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":78,"threads_started":1}
I20260806 21:15:32.750880 11152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:32.750461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.3:39619)
0806 21:15:32.750724 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:32.750778 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":97,"threads_started":1}
I20260806 21:15:33.426753 11153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:33.426282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.1:49741)
0806 21:15:33.426584 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:33.426653 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":100,"threads_started":1}
I20260806 21:15:33.550403 11155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:33.549909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.2:44717)
0806 21:15:33.550221 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:33.550287 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":103,"threads_started":1}
I20260806 21:15:33.752162 11158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:33.751651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.3:39463)
0806 21:15:33.751937 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:33.752002 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":80,"threads_started":1}
I20260806 21:15:34.428153 11159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:34.427670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.1:57301)
0806 21:15:34.427933 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:34.427995 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":70,"threads_started":1}
I20260806 21:15:34.551908 11160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:34.551404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.2:41203)
0806 21:15:34.551719 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:34.551790 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":92,"threads_started":1}
I20260806 21:15:34.752848 11162 raft_consensus.cc:482] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:15:34.753008 11162 raft_consensus.cc:504] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6da22627b28b4b19bef57184d3167576" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 46401 } } peers { permanent_uuid: "1522aa3f9fe54b1d95e66ff625693695" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 37533 } } peers { permanent_uuid: "f0dda022ce3140e08642813b191a3d19" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36913 } }
I20260806 21:15:34.753399 11165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:34.752979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.3:50849)
0806 21:15:34.753232 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:34.753294 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":70,"threads_started":1}
I20260806 21:15:34.753476 11162 leader_election.cc:288] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6da22627b28b4b19bef57184d3167576 (127.16.178.3:46401), 1522aa3f9fe54b1d95e66ff625693695 (127.16.178.2:37533)
W20260806 21:15:34.754254 10543 leader_election.cc:341] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6da22627b28b4b19bef57184d3167576 (127.16.178.3:46401): Not found: Tablet not found: 2b5f64c69b4245de92fc32b1a711faa7
W20260806 21:15:34.754364 10544 leader_election.cc:341] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1522aa3f9fe54b1d95e66ff625693695 (127.16.178.2:37533): Not found: Tablet not found: 2b5f64c69b4245de92fc32b1a711faa7
I20260806 21:15:34.754464 10544 leader_election.cc:302] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f0dda022ce3140e08642813b191a3d19; no voters: 1522aa3f9fe54b1d95e66ff625693695, 6da22627b28b4b19bef57184d3167576
I20260806 21:15:34.754621 11162 raft_consensus.cc:2726] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 21:15:35.429533 11166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:35.429073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.1:49415)
0806 21:15:35.429349 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:35.429418 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":86,"threads_started":1}
I20260806 21:15:35.553380 11167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:35.552862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.2:36213)
0806 21:15:35.553196 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:35.553269 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":84,"threads_started":1}
I20260806 21:15:35.754825 11168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:35.754386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.3:56179)
0806 21:15:35.754663 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:35.754722 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":82,"threads_started":1}
I20260806 21:15:36.430845 11169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:36.430337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.1:38403)
0806 21:15:36.430673 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:36.430737 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":116,"threads_started":1}
I20260806 21:15:36.554738 11172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:36.554274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.2:60561)
0806 21:15:36.554568 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:36.554631 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":107,"threads_started":1}
I20260806 21:15:36.756273 11173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:36.755707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.3:37919)
0806 21:15:36.756044 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:36.756163 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":95,"threads_started":1}
I20260806 21:15:37.432261 11174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:37.431711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.1:38177)
0806 21:15:37.432035 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:37.432153 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":98,"threads_started":1}
I20260806 21:15:37.556196 11175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:37.555645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.2:60595)
0806 21:15:37.555974 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:37.556047 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":107,"threads_started":1}
I20260806 21:15:37.757709 11178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:37.757213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.3:60737)
0806 21:15:37.757524 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:37.757599 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":87,"threads_started":1}
I20260806 21:15:38.433729 11179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:38.433142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.1:60029)
0806 21:15:38.433490 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:38.433586 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":88,"threads_started":1}
I20260806 21:15:38.557588 11180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:38.557082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.2:55487)
0806 21:15:38.557379 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:38.557464 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":95,"threads_started":1}
I20260806 21:15:38.759104 11181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:38.758616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.3:57861)
0806 21:15:38.758910 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:38.758983 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":89,"threads_started":1}
I20260806 21:15:39.291471 11182 raft_consensus.cc:482] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:15:39.291632 11182 raft_consensus.cc:504] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6da22627b28b4b19bef57184d3167576" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 46401 } } peers { permanent_uuid: "1522aa3f9fe54b1d95e66ff625693695" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 37533 } } peers { permanent_uuid: "f0dda022ce3140e08642813b191a3d19" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 36913 } }
I20260806 21:15:39.292131 11182 leader_election.cc:288] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6da22627b28b4b19bef57184d3167576 (127.16.178.3:46401), 1522aa3f9fe54b1d95e66ff625693695 (127.16.178.2:37533)
W20260806 21:15:39.292915 10544 leader_election.cc:341] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1522aa3f9fe54b1d95e66ff625693695 (127.16.178.2:37533): Not found: Tablet not found: 2b5f64c69b4245de92fc32b1a711faa7
W20260806 21:15:39.293040 10543 leader_election.cc:341] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6da22627b28b4b19bef57184d3167576 (127.16.178.3:46401): Not found: Tablet not found: 2b5f64c69b4245de92fc32b1a711faa7
I20260806 21:15:39.293135 10543 leader_election.cc:302] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f0dda022ce3140e08642813b191a3d19; no voters: 1522aa3f9fe54b1d95e66ff625693695, 6da22627b28b4b19bef57184d3167576
I20260806 21:15:39.293282 11182 raft_consensus.cc:2726] T 2b5f64c69b4245de92fc32b1a711faa7 P f0dda022ce3140e08642813b191a3d19 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 21:15:39.435380 11183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:39.434897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.1:45359)
0806 21:15:39.435210 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:39.435279 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":101,"threads_started":1}
I20260806 21:15:39.559139 11185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:39.558627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.2:33505)
0806 21:15:39.558945 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:39.559010 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":96,"threads_started":1}
I20260806 21:15:39.749836 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 10463
I20260806 21:15:39.762039 11187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:39.761531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:37115 (local address 127.16.178.3:48967)
0806 21:15:39.761855 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:39.761930 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.62:37115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":85,"threads_started":1}
I20260806 21:15:39.768543 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 10705
I20260806 21:15:39.779263 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 10916
2026-08-06T21:15:39Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59449 ms)
[----------] 30 tests from SecurityITest (252951 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:15:39 dist-test-slave-lgr5 krb5kdc[11191](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:15:39 dist-test-slave-lgr5 krb5kdc[11191](info): set up 2 sockets
Aug 06 21:15:39 dist-test-slave-lgr5 krb5kdc[11191](info): commencing operation
krb5kdc: starting...
W20260806 21:15:40.957463 17096 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Aug 06 21:15:41 dist-test-slave-lgr5 krb5kdc[11191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050941, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:15:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:15:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.178.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
I20260806 21:15:41.103580 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:35091
--webserver_interface=127.16.178.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:39743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:35091
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:15:41.220860 11208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:15:41.221138 11208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:15:41.221177 11208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:15:41.225186 11208 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:15:41.225250 11208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:15:41.225268 11208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:15:41.225284 11208 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:15:41.225299 11208 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:15:41.230939 11208 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:39743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:35091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:35091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.178.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:15:41.232113 11208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:15:41.233150 11208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:15:41.242286 11214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:41.242311 11215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:41.242359 11218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:41.242378 11208 server_base.cc:1034] running on GCE node
I20260806 21:15:41.243216 11208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:15:41.244549 11208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:15:41.245731 11208 hybrid_clock.cc:648] HybridClock initialized: now 1786050941245703 us; error 48 us; skew 500 ppm
Aug 06 21:15:41 dist-test-slave-lgr5 krb5kdc[11191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050941, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:15:41.248574 11208 init.cc:377] Logged in from keytab as kudu/127.16.178.62@KRBTEST.COM (short username kudu)
I20260806 21:15:41.249470 11208 webserver.cc:460] Webserver started at http://127.16.178.62:39985/ using document root <none> and password file <none>
I20260806 21:15:41.250022 11208 fs_manager.cc:359] Metadata directory not provided
I20260806 21:15:41.250083 11208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:15:41.250278 11208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:15:41.252434 11208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "de0a6385cb3142059c77e7a033717094"
format_stamp: "Formatted at 2026-08-06 21:15:41 on dist-test-slave-lgr5"
server_key: "e03c71d063205650fb52941f2fe7eccc"
server_key_iv: "ab3a32ae875390622dd5e643fccece0c"
server_key_version: "encryptionkey@0"
I20260806 21:15:41.253053 11208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "de0a6385cb3142059c77e7a033717094"
format_stamp: "Formatted at 2026-08-06 21:15:41 on dist-test-slave-lgr5"
server_key: "e03c71d063205650fb52941f2fe7eccc"
server_key_iv: "ab3a32ae875390622dd5e643fccece0c"
server_key_version: "encryptionkey@0"
I20260806 21:15:41.256843 11208 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 21:15:41.259375 11227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:41.260679 11208 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 21:15:41.260813 11208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "de0a6385cb3142059c77e7a033717094"
format_stamp: "Formatted at 2026-08-06 21:15:41 on dist-test-slave-lgr5"
server_key: "e03c71d063205650fb52941f2fe7eccc"
server_key_iv: "ab3a32ae875390622dd5e643fccece0c"
server_key_version: "encryptionkey@0"
I20260806 21:15:41.260926 11208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:15:41.286206 11208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:15:41.289871 11208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:15:41.290066 11208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:15:41.297953 11208 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.62:35091
I20260806 21:15:41.297976 11302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.62:35091 every 8 connection(s)
I20260806 21:15:41.299011 11208 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:15:41.301664 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 11208
I20260806 21:15:41.301810 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:15:41.302163 11303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:15:41.302171 17096 external_mini_cluster.cc:1428] Setting key ca165bfa490a7c7ad178be3505cdc6e6
I20260806 21:15:41.309934 11303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de0a6385cb3142059c77e7a033717094: Bootstrap starting.
Aug 06 21:15:41 dist-test-slave-lgr5 krb5kdc[11191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050941, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:15:41.312948 11303 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de0a6385cb3142059c77e7a033717094: Neither blocks nor log segments found. Creating new log.
I20260806 21:15:41.313830 11303 log.cc:824] T 00000000000000000000000000000000 P de0a6385cb3142059c77e7a033717094: Log is configured to *not* fsync() on all Append() calls
I20260806 21:15:41.316048 11303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de0a6385cb3142059c77e7a033717094: No bootstrap required, opened a new log
I20260806 21:15:41.318012 11308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:41.303659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58558 (local address 127.16.178.62:35091)
0806 21:15:41.304137 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:41.304155 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:41.304195 (+ 40us) server_negotiation.cc:407] Connection header received
0806 21:15:41.305212 (+ 1017us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:41.305235 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:41.305626 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:41.305937 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:15:41.306930 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:41.308019 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:41.309375 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:41.309920 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:41.310062 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:41.312969 (+ 2907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:15:41.312974 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:15:41.312989 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:15:41.313021 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:15:41.315001 (+ 1980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:41.315487 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:41.315495 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:41.315499 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:41.315625 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:41.315928 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:41.315932 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:41.315933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:41.316425 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:15:41.316619 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:41.316943 (+ 324us) server_negotiation.cc:299] Negotiation successful
0806 21:15:41.317253 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":122,"threads_started":1}
I20260806 21:15:41.319780 11303 raft_consensus.cc:348] T 00000000000000000000000000000000 P de0a6385cb3142059c77e7a033717094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de0a6385cb3142059c77e7a033717094" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 35091 } }
I20260806 21:15:41.320015 11303 raft_consensus.cc:374] T 00000000000000000000000000000000 P de0a6385cb3142059c77e7a033717094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:15:41.320099 11303 raft_consensus.cc:729] T 00000000000000000000000000000000 P de0a6385cb3142059c77e7a033717094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de0a6385cb3142059c77e7a033717094, State: Initialized, Role: FOLLOWER
I20260806 21:15:41.320566 11303 consensus_queue.cc:260] T 00000000000000000000000000000000 P de0a6385cb3142059c77e7a033717094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de0a6385cb3142059c77e7a033717094" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 35091 } }
I20260806 21:15:41.320724 11303 raft_consensus.cc:388] T 00000000000000000000000000000000 P de0a6385cb3142059c77e7a033717094 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:15:41.320788 11303 raft_consensus.cc:482] T 00000000000000000000000000000000 P de0a6385cb3142059c77e7a033717094 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:15:41.320886 11303 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de0a6385cb3142059c77e7a033717094 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:15:41.321808 11303 raft_consensus.cc:504] T 00000000000000000000000000000000 P de0a6385cb3142059c77e7a033717094 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de0a6385cb3142059c77e7a033717094" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 35091 } }
I20260806 21:15:41.322139 11303 leader_election.cc:302] T 00000000000000000000000000000000 P de0a6385cb3142059c77e7a033717094 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de0a6385cb3142059c77e7a033717094; no voters:
I20260806 21:15:41.322454 11303 leader_election.cc:288] T 00000000000000000000000000000000 P de0a6385cb3142059c77e7a033717094 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:15:41.322558 11310 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de0a6385cb3142059c77e7a033717094 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:15:41.322785 11310 raft_consensus.cc:686] T 00000000000000000000000000000000 P de0a6385cb3142059c77e7a033717094 [term 1 LEADER]: Becoming Leader. State: Replica: de0a6385cb3142059c77e7a033717094, State: Running, Role: LEADER
I20260806 21:15:41.323128 11310 consensus_queue.cc:237] T 00000000000000000000000000000000 P de0a6385cb3142059c77e7a033717094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de0a6385cb3142059c77e7a033717094" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 35091 } }
I20260806 21:15:41.323724 11303 sys_catalog.cc:564] T 00000000000000000000000000000000 P de0a6385cb3142059c77e7a033717094 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:15:41.324832 11312 sys_catalog.cc:455] T 00000000000000000000000000000000 P de0a6385cb3142059c77e7a033717094 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de0a6385cb3142059c77e7a033717094" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de0a6385cb3142059c77e7a033717094" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 35091 } } }
I20260806 21:15:41.324882 11316 sys_catalog.cc:455] T 00000000000000000000000000000000 P de0a6385cb3142059c77e7a033717094 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de0a6385cb3142059c77e7a033717094. Latest consensus state: current_term: 1 leader_uuid: "de0a6385cb3142059c77e7a033717094" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de0a6385cb3142059c77e7a033717094" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 35091 } } }
I20260806 21:15:41.325006 11312 sys_catalog.cc:458] T 00000000000000000000000000000000 P de0a6385cb3142059c77e7a033717094 [sys.catalog]: This master's current role is: LEADER
I20260806 21:15:41.325006 11316 sys_catalog.cc:458] T 00000000000000000000000000000000 P de0a6385cb3142059c77e7a033717094 [sys.catalog]: This master's current role is: LEADER
I20260806 21:15:41.325372 11325 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:15:41.328665 11325 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:15:41.334396 11325 catalog_manager.cc:1347] Generated new cluster ID: 4dc7b15b197f464383abd0fadb4b8fa7
I20260806 21:15:41.334476 11325 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:15:41.340338 11325 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:15:41.341956 11325 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:15:41.356261 11325 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de0a6385cb3142059c77e7a033717094: Generated new TSK 0
I20260806 21:15:41.357040 11325 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.178.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab.
Entry for principal kudu/127.16.178.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab.
I20260806 21:15:41.391662 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.1:0
--local_ip_for_outbound_sockets=127.16.178.1
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:35091
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab
--principal=kudu/127.16.178.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:39743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:15:41.539103 11345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:15:41.539386 11345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:15:41.539428 11345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:15:41.543550 11345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:15:41.543623 11345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:15:41.543695 11345 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.1
I20260806 21:15:41.549327 11345 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:39743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab
--principal=kudu/127.16.178.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:35091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.1
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:15:41.550581 11345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:15:41.551641 11345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:15:41.561537 11350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:41.561537 11356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:41.561554 11345 server_base.cc:1034] running on GCE node
W20260806 21:15:41.561554 11352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:41.562203 11345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:15:41.562909 11345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:15:41.564065 11345 hybrid_clock.cc:648] HybridClock initialized: now 1786050941564051 us; error 32 us; skew 500 ppm
Aug 06 21:15:41 dist-test-slave-lgr5 krb5kdc[11191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050941, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:15:41.567060 11345 init.cc:377] Logged in from keytab as kudu/127.16.178.1@KRBTEST.COM (short username kudu)
I20260806 21:15:41.568048 11345 webserver.cc:460] Webserver started at http://127.16.178.1:33883/ using document root <none> and password file <none>
I20260806 21:15:41.568660 11345 fs_manager.cc:359] Metadata directory not provided
I20260806 21:15:41.568723 11345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:15:41.568907 11345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:15:41.571097 11345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-0/data/instance:
uuid: "f3f46e16e5cc440bb077a4c9200b5b40"
format_stamp: "Formatted at 2026-08-06 21:15:41 on dist-test-slave-lgr5"
server_key: "99df5c10815e17355ab4ff1b6c3ad9d1"
server_key_iv: "bfa112232993970ca6a4775841746c60"
server_key_version: "encryptionkey@0"
I20260806 21:15:41.571642 11345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance:
uuid: "f3f46e16e5cc440bb077a4c9200b5b40"
format_stamp: "Formatted at 2026-08-06 21:15:41 on dist-test-slave-lgr5"
server_key: "99df5c10815e17355ab4ff1b6c3ad9d1"
server_key_iv: "bfa112232993970ca6a4775841746c60"
server_key_version: "encryptionkey@0"
I20260806 21:15:41.575668 11345 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 21:15:41.578420 11363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:41.579725 11345 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 21:15:41.579859 11345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-0/wal
uuid: "f3f46e16e5cc440bb077a4c9200b5b40"
format_stamp: "Formatted at 2026-08-06 21:15:41 on dist-test-slave-lgr5"
server_key: "99df5c10815e17355ab4ff1b6c3ad9d1"
server_key_iv: "bfa112232993970ca6a4775841746c60"
server_key_version: "encryptionkey@0"
I20260806 21:15:41.579988 11345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:15:41.597651 11345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:15:41.601292 11345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:15:41.601475 11345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:15:41.602077 11345 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:15:41.602967 11345 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:15:41.603018 11345 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:41.603060 11345 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:15:41.603080 11345 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:41.612283 11345 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.1:33135
I20260806 21:15:41.612317 11513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.1:33135 every 8 connection(s)
I20260806 21:15:41.613262 11345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
I20260806 21:15:41.621248 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 11345
I20260806 21:15:41.621380 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance
I20260806 21:15:41.621657 17096 external_mini_cluster.cc:1428] Setting key b3f5763aab743d1f709ed5314610f3fb
Aug 06 21:15:41 dist-test-slave-lgr5 krb5kdc[11191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050941, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:15:41.627504 11308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:41.615158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:33831 (local address 127.16.178.62:35091)
0806 21:15:41.615314 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:41.615318 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:41.616315 (+ 997us) server_negotiation.cc:407] Connection header received
0806 21:15:41.617636 (+ 1321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:41.617640 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:41.617701 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:41.617776 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:15:41.619010 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:41.619558 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:41.620772 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:41.621046 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:41.621160 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:41.623806 (+ 2646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:15:41.623809 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:15:41.623811 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:15:41.623835 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:15:41.625362 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:41.625879 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:41.625883 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:41.625885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:41.625976 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:41.626270 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:41.626273 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:41.626275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:41.626517 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:15:41.626618 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:41.627208 (+ 590us) server_negotiation.cc:299] Negotiation successful
0806 21:15:41.627339 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 21:15:41.628427 11519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:41.615601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35091 (local address 127.16.178.1:33831)
0806 21:15:41.616159 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:41.616207 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:41.617415 (+ 1208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:41.617979 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:41.617986 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:41.618353 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:15:41.618848 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:41.618862 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:41.619680 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:41.619683 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:41.620629 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:41.620640 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:41.621226 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:41.621229 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:41.621425 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:41.622017 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:15:41.622033 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:15:41.623641 (+ 1608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:15:41.625491 (+ 1850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:15:41.625497 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:15:41.625511 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:15:41.625771 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:15:41.626054 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:15:41.626056 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:15:41.626058 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:15:41.626191 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:15:41.626651 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:15:41.626658 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:15:41.626950 (+ 292us) client_negotiation.cc:769] Sending connection context
0806 21:15:41.627131 (+ 181us) client_negotiation.cc:240] Negotiation successful
0806 21:15:41.627427 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":116,"threads_started":1}
I20260806 21:15:41.629895 11517 heartbeater.cc:347] Connected to a master server at 127.16.178.62:35091
I20260806 21:15:41.630208 11517 heartbeater.cc:464] Registering TS with master...
I20260806 21:15:41.630901 11517 heartbeater.cc:511] Master 127.16.178.62:35091 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.16.178.2@KRBTEST.COM; defaulting to no policy
I20260806 21:15:41.632732 11250 ts_manager.cc:194] Registered new tserver with Master: f3f46e16e5cc440bb077a4c9200b5b40 (127.16.178.1:33135)
I20260806 21:15:41.633996 11250 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.1@KRBTEST.COM'} at 127.16.178.1:33831
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab.
Entry for principal kudu/127.16.178.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab.
I20260806 21:15:41.654786 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.2:0
--local_ip_for_outbound_sockets=127.16.178.2
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:35091
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab
--principal=kudu/127.16.178.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:39743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:15:41.769336 11524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:15:41.769646 11524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:15:41.769691 11524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:15:41.773865 11524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:15:41.773941 11524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:15:41.774003 11524 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.2
I20260806 21:15:41.779433 11524 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:39743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab
--principal=kudu/127.16.178.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:35091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.2
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:15:41.780603 11524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:15:41.781651 11524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:15:41.791462 11532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:41.791471 11533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:41.791533 11524 server_base.cc:1034] running on GCE node
W20260806 21:15:41.791728 11535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:41.792157 11524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:15:41.792793 11524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:15:41.793962 11524 hybrid_clock.cc:648] HybridClock initialized: now 1786050941793937 us; error 42 us; skew 500 ppm
Aug 06 21:15:41 dist-test-slave-lgr5 krb5kdc[11191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050941, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:15:41.796710 11524 init.cc:377] Logged in from keytab as kudu/127.16.178.2@KRBTEST.COM (short username kudu)
I20260806 21:15:41.797581 11524 webserver.cc:460] Webserver started at http://127.16.178.2:44725/ using document root <none> and password file <none>
I20260806 21:15:41.798141 11524 fs_manager.cc:359] Metadata directory not provided
I20260806 21:15:41.798209 11524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:15:41.798416 11524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:15:41.800522 11524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-1/data/instance:
uuid: "3a7430b190894319a7158be1bd398cbb"
format_stamp: "Formatted at 2026-08-06 21:15:41 on dist-test-slave-lgr5"
server_key: "dced1c0590ac0db1251c06d1dc111528"
server_key_iv: "567a84bf77aad2324ba317e4cf5ecb86"
server_key_version: "encryptionkey@0"
I20260806 21:15:41.801071 11524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance:
uuid: "3a7430b190894319a7158be1bd398cbb"
format_stamp: "Formatted at 2026-08-06 21:15:41 on dist-test-slave-lgr5"
server_key: "dced1c0590ac0db1251c06d1dc111528"
server_key_iv: "567a84bf77aad2324ba317e4cf5ecb86"
server_key_version: "encryptionkey@0"
I20260806 21:15:41.804800 11524 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:15:41.807168 11547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:41.808424 11524 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 21:15:41.808547 11524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-1/wal
uuid: "3a7430b190894319a7158be1bd398cbb"
format_stamp: "Formatted at 2026-08-06 21:15:41 on dist-test-slave-lgr5"
server_key: "dced1c0590ac0db1251c06d1dc111528"
server_key_iv: "567a84bf77aad2324ba317e4cf5ecb86"
server_key_version: "encryptionkey@0"
I20260806 21:15:41.808661 11524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:15:41.825419 11524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:15:41.828882 11524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:15:41.829088 11524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:15:41.829726 11524 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:15:41.830641 11524 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:15:41.830696 11524 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:41.830760 11524 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:15:41.830785 11524 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:41.840059 11524 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.2:45847
I20260806 21:15:41.840118 11718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.2:45847 every 8 connection(s)
I20260806 21:15:41.841090 11524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
I20260806 21:15:41.842408 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 11524
I20260806 21:15:41.842547 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance
I20260806 21:15:41.842880 17096 external_mini_cluster.cc:1428] Setting key f6c7362fba86279b0f362cfbf63b3f02
Aug 06 21:15:41 dist-test-slave-lgr5 krb5kdc[11191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050941, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.2@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.178.3@KRBTEST.COM; defaulting to no policy
I20260806 21:15:41.855868 11308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:41.843076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:38589 (local address 127.16.178.62:35091)
0806 21:15:41.843214 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:41.843218 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:41.844359 (+ 1141us) server_negotiation.cc:407] Connection header received
0806 21:15:41.845704 (+ 1345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:41.845708 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:41.845763 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:41.845850 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:15:41.847015 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:41.847784 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:41.849473 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:41.849695 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:41.849792 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:41.852463 (+ 2671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:15:41.852467 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:15:41.852469 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:15:41.852495 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:15:41.853824 (+ 1329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:41.854292 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:41.854294 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:41.854295 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:41.854361 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:41.854676 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:41.854679 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:41.854681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:41.854873 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:15:41.854965 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:41.855522 (+ 557us) server_negotiation.cc:299] Negotiation successful
0806 21:15:41.855655 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.3@KRBTEST.COM" created.
I20260806 21:15:41.856827 11723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:41.843618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35091 (local address 127.16.178.2:38589)
0806 21:15:41.844216 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:41.844273 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:41.845488 (+ 1215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:41.846013 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:41.846020 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:41.846371 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:15:41.846858 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:41.846871 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:41.847943 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:41.847947 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:41.849326 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:41.849340 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:41.849866 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:41.849871 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:41.850044 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:41.850633 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:15:41.850649 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:15:41.852272 (+ 1623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:15:41.853934 (+ 1662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:15:41.853939 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:15:41.853953 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:15:41.854209 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:15:41.854420 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:15:41.854422 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:15:41.854424 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:15:41.854581 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:15:41.854974 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:15:41.854979 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:15:41.855240 (+ 261us) client_negotiation.cc:769] Sending connection context
0806 21:15:41.855413 (+ 173us) client_negotiation.cc:240] Negotiation successful
0806 21:15:41.855760 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":135,"threads_started":1}
I20260806 21:15:41.858184 11720 heartbeater.cc:347] Connected to a master server at 127.16.178.62:35091
I20260806 21:15:41.858425 11720 heartbeater.cc:464] Registering TS with master...
I20260806 21:15:41.858958 11720 heartbeater.cc:511] Master 127.16.178.62:35091 requested a full tablet report, sending...
I20260806 21:15:41.860108 11250 ts_manager.cc:194] Registered new tserver with Master: 3a7430b190894319a7158be1bd398cbb (127.16.178.2:45847)
I20260806 21:15:41.860695 11250 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.2@KRBTEST.COM'} at 127.16.178.2:38589
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab.
Entry for principal kudu/127.16.178.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab.
I20260806 21:15:41.873387 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.3:0
--local_ip_for_outbound_sockets=127.16.178.3
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:35091
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab
--principal=kudu/127.16.178.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:39743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:15:41.993221 11727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:15:41.993561 11727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:15:41.993626 11727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:15:41.997805 11727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:15:41.997880 11727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:15:41.997960 11727 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.3
I20260806 21:15:42.003614 11727 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:39743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab
--principal=kudu/127.16.178.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:35091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.3
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:15:42.004878 11727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:15:42.006080 11727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:15:42.016911 11740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:42.016953 11737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:42.016912 11735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:42.017310 11727 server_base.cc:1034] running on GCE node
I20260806 21:15:42.017937 11727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:15:42.018697 11727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:15:42.019917 11727 hybrid_clock.cc:648] HybridClock initialized: now 1786050942019887 us; error 47 us; skew 500 ppm
Aug 06 21:15:42 dist-test-slave-lgr5 krb5kdc[11191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050942, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:15:42.023154 11727 init.cc:377] Logged in from keytab as kudu/127.16.178.3@KRBTEST.COM (short username kudu)
I20260806 21:15:42.024432 11727 webserver.cc:460] Webserver started at http://127.16.178.3:34249/ using document root <none> and password file <none>
I20260806 21:15:42.025146 11727 fs_manager.cc:359] Metadata directory not provided
I20260806 21:15:42.025242 11727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:15:42.025486 11727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:15:42.027909 11727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-2/data/instance:
uuid: "b3c0a17350a74bff8a9640e5cf0625ac"
format_stamp: "Formatted at 2026-08-06 21:15:42 on dist-test-slave-lgr5"
server_key: "fa83a00ab56f01248334af3a04e4c60b"
server_key_iv: "d09dc98577d6bbc88140cfd574c0f066"
server_key_version: "encryptionkey@0"
I20260806 21:15:42.028631 11727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance:
uuid: "b3c0a17350a74bff8a9640e5cf0625ac"
format_stamp: "Formatted at 2026-08-06 21:15:42 on dist-test-slave-lgr5"
server_key: "fa83a00ab56f01248334af3a04e4c60b"
server_key_iv: "d09dc98577d6bbc88140cfd574c0f066"
server_key_version: "encryptionkey@0"
I20260806 21:15:42.032905 11727 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:15:42.035606 11753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:42.036952 11727 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:15:42.037086 11727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-2/wal
uuid: "b3c0a17350a74bff8a9640e5cf0625ac"
format_stamp: "Formatted at 2026-08-06 21:15:42 on dist-test-slave-lgr5"
server_key: "fa83a00ab56f01248334af3a04e4c60b"
server_key_iv: "d09dc98577d6bbc88140cfd574c0f066"
server_key_version: "encryptionkey@0"
I20260806 21:15:42.037205 11727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:15:42.054867 11727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:15:42.058063 11727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:15:42.058257 11727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:15:42.058871 11727 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:15:42.059741 11727 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:15:42.059791 11727 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:42.059847 11727 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:15:42.059876 11727 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:42.069033 11727 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.3:40709
I20260806 21:15:42.069069 11908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.3:40709 every 8 connection(s)
I20260806 21:15:42.069975 11727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
I20260806 21:15:42.071075 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 11727
I20260806 21:15:42.071214 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance
I20260806 21:15:42.071493 17096 external_mini_cluster.cc:1428] Setting key d0a98a209f452b0ea91e85102eceec21
Aug 06 21:15:42 dist-test-slave-lgr5 krb5kdc[11191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050942, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.3@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:15:42.084275 11308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:42.072010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:43503 (local address 127.16.178.62:35091)
0806 21:15:42.072177 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:42.072181 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:42.073262 (+ 1081us) server_negotiation.cc:407] Connection header received
0806 21:15:42.074757 (+ 1495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:42.074760 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:42.074808 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:42.074875 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:15:42.076105 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:42.076648 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:42.077809 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:42.078039 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:42.078116 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:42.080569 (+ 2453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:15:42.080573 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:15:42.080575 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:15:42.080603 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:15:42.081888 (+ 1285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:42.082432 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:42.082435 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:42.082437 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:42.082535 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:42.082864 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:42.082867 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:42.082868 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:42.083120 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:15:42.083211 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:42.083874 (+ 663us) server_negotiation.cc:299] Negotiation successful
0806 21:15:42.084062 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 21:15:42.085350 11912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:42.072549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:35091 (local address 127.16.178.3:43503)
0806 21:15:42.073100 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:42.073166 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:42.074532 (+ 1366us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:42.075044 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:42.075052 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:42.075425 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:15:42.075941 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:42.075956 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:42.076771 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:42.076774 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:42.077686 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:42.077696 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:42.078176 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:42.078179 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:42.078330 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:42.078876 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:15:42.078893 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:15:42.080390 (+ 1497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:15:42.082012 (+ 1622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:15:42.082018 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:15:42.082034 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:15:42.082311 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:15:42.082620 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:15:42.082622 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:15:42.082623 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:15:42.082773 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:15:42.083261 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:15:42.083268 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:15:42.083604 (+ 336us) client_negotiation.cc:769] Sending connection context
0806 21:15:42.083792 (+ 188us) client_negotiation.cc:240] Negotiation successful
0806 21:15:42.084242 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":128,"threads_started":1}
I20260806 21:15:42.086488 11909 heartbeater.cc:347] Connected to a master server at 127.16.178.62:35091
I20260806 21:15:42.086748 11909 heartbeater.cc:464] Registering TS with master...
I20260806 21:15:42.087245 11909 heartbeater.cc:511] Master 127.16.178.62:35091 requested a full tablet report, sending...
I20260806 21:15:42.088274 11250 ts_manager.cc:194] Registered new tserver with Master: b3c0a17350a74bff8a9640e5cf0625ac (127.16.178.3:40709)
I20260806 21:15:42.088831 11250 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.3@KRBTEST.COM'} at 127.16.178.3:43503
I20260806 21:15:42.097914 17096 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 21:15:42.107103 11308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:42.100002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58574 (local address 127.16.178.62:35091)
0806 21:15:42.100204 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:42.100209 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:42.100319 (+ 110us) server_negotiation.cc:407] Connection header received
0806 21:15:42.100443 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:42.100445 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:42.100490 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:42.100551 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:15:42.101206 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:42.101732 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:42.102522 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:42.102723 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:42.102788 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:42.104126 (+ 1338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:15:42.104131 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:15:42.104134 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:15:42.104166 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:15:42.105477 (+ 1311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:42.105920 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:42.105923 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:42.105925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:42.106015 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:42.106264 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:42.106266 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:42.106267 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:42.106519 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:15:42.106633 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:42.106836 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 21:15:42.106940 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260806 21:15:42.110450 11250 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58574:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:15:42.114073 11250 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:15:42.131099 11923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:42.124333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.3:40709 (local address 127.0.0.1:36074)
0806 21:15:42.125289 (+ 956us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:42.125341 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:42.125501 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:42.126334 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:42.126337 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:42.126350 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:15:42.126515 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:42.126522 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:42.128027 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:42.128030 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:42.129319 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:42.129330 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:42.130452 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:42.130456 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:42.130616 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:42.130708 (+ 92us) client_negotiation.cc:769] Sending connection context
0806 21:15:42.130781 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 21:15:42.130934 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":815,"mutex_wait_us":109,"spinlock_wait_cycles":8704,"thread_start_us":112,"threads_started":1}
I20260806 21:15:42.131153 11928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:42.124899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36074 (local address 127.16.178.3:40709)
0806 21:15:42.125708 (+ 809us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:42.125761 (+ 53us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:42.125785 (+ 24us) server_negotiation.cc:407] Connection header received
0806 21:15:42.125864 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:42.125869 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:42.126195 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:42.126338 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:15:42.126654 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:42.127880 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:42.129453 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:42.130304 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:42.130686 (+ 382us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:42.130843 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:42.130902 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 21:15:42.130997 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":706,"thread_start_us":162,"threads_started":1}
I20260806 21:15:42.131153 11930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:42.125062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58780 (local address 127.16.178.2:45847)
0806 21:15:42.125784 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:42.125803 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:42.126181 (+ 378us) server_negotiation.cc:407] Connection header received
0806 21:15:42.126258 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:42.126263 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:42.126433 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:42.126573 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:15:42.126792 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:42.127538 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:42.129464 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:42.130299 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:42.130415 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:42.130570 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:42.130904 (+ 334us) server_negotiation.cc:299] Negotiation successful
0806 21:15:42.130997 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":133,"threads_started":1}
I20260806 21:15:42.131959 11926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:42.124197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:33135 (local address 127.0.0.1:36492)
0806 21:15:42.125290 (+ 1093us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:42.125342 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:42.125500 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:42.126283 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:42.126290 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:42.126315 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:15:42.126514 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:42.126522 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:42.129455 (+ 2933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:42.129458 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:42.130520 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:42.130532 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:42.131559 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:42.131563 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:42.131652 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:42.131677 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 21:15:42.131752 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 21:15:42.131820 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":913,"mutex_wait_us":6,"spinlock_wait_cycles":6656,"thread_start_us":124,"threads_started":1}
I20260806 21:15:42.132493 11925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:42.124341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36492 (local address 127.16.178.1:33135)
0806 21:15:42.125065 (+ 724us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:42.125081 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:42.125431 (+ 350us) server_negotiation.cc:407] Connection header received
0806 21:15:42.125697 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:42.125703 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:42.125943 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:42.126103 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:15:42.126654 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:42.127880 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:42.130652 (+ 2772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:42.131431 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:42.131538 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:42.131678 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:42.132187 (+ 509us) server_negotiation.cc:299] Negotiation successful
0806 21:15:42.132292 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":126,"threads_started":1}
I20260806 21:15:42.132278 11922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:42.124555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.2:45847 (local address 127.0.0.1:58780)
0806 21:15:42.125997 (+ 1442us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:42.126010 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:42.126096 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:42.126568 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:42.126571 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:42.126587 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:15:42.126680 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:42.126687 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:42.127659 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:42.127663 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:42.129319 (+ 1656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:42.129330 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:42.130452 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:42.130456 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:42.130607 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:42.130680 (+ 73us) client_negotiation.cc:769] Sending connection context
0806 21:15:42.130798 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 21:15:42.132153 (+ 1355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1330,"mutex_wait_us":1164,"thread_start_us":112,"threads_started":1}
I20260806 21:15:42.134640 11814 tablet_service.cc:1467] Processing CreateTablet for tablet 16a88a37ca60484f881842676f08620d (DEFAULT_TABLE table=test-table [id=fb5e494943c44cc5b150554b7c75070e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:15:42.135739 11814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16a88a37ca60484f881842676f08620d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:15:42.135406 11422 tablet_service.cc:1467] Processing CreateTablet for tablet 16a88a37ca60484f881842676f08620d (DEFAULT_TABLE table=test-table [id=fb5e494943c44cc5b150554b7c75070e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:15:42.135406 11619 tablet_service.cc:1467] Processing CreateTablet for tablet 16a88a37ca60484f881842676f08620d (DEFAULT_TABLE table=test-table [id=fb5e494943c44cc5b150554b7c75070e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:15:42.136439 11422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16a88a37ca60484f881842676f08620d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:15:42.136439 11619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16a88a37ca60484f881842676f08620d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:15:42.143496 11933 tablet_bootstrap.cc:492] T 16a88a37ca60484f881842676f08620d P b3c0a17350a74bff8a9640e5cf0625ac: Bootstrap starting.
I20260806 21:15:42.144112 11935 tablet_bootstrap.cc:492] T 16a88a37ca60484f881842676f08620d P f3f46e16e5cc440bb077a4c9200b5b40: Bootstrap starting.
I20260806 21:15:42.144109 11934 tablet_bootstrap.cc:492] T 16a88a37ca60484f881842676f08620d P 3a7430b190894319a7158be1bd398cbb: Bootstrap starting.
I20260806 21:15:42.145725 11933 tablet_bootstrap.cc:654] T 16a88a37ca60484f881842676f08620d P b3c0a17350a74bff8a9640e5cf0625ac: Neither blocks nor log segments found. Creating new log.
I20260806 21:15:42.146265 11935 tablet_bootstrap.cc:654] T 16a88a37ca60484f881842676f08620d P f3f46e16e5cc440bb077a4c9200b5b40: Neither blocks nor log segments found. Creating new log.
I20260806 21:15:42.146265 11934 tablet_bootstrap.cc:654] T 16a88a37ca60484f881842676f08620d P 3a7430b190894319a7158be1bd398cbb: Neither blocks nor log segments found. Creating new log.
I20260806 21:15:42.146957 11933 log.cc:824] T 16a88a37ca60484f881842676f08620d P b3c0a17350a74bff8a9640e5cf0625ac: Log is configured to *not* fsync() on all Append() calls
I20260806 21:15:42.147130 11935 log.cc:824] T 16a88a37ca60484f881842676f08620d P f3f46e16e5cc440bb077a4c9200b5b40: Log is configured to *not* fsync() on all Append() calls
I20260806 21:15:42.147413 11934 log.cc:824] T 16a88a37ca60484f881842676f08620d P 3a7430b190894319a7158be1bd398cbb: Log is configured to *not* fsync() on all Append() calls
I20260806 21:15:42.149296 11933 tablet_bootstrap.cc:492] T 16a88a37ca60484f881842676f08620d P b3c0a17350a74bff8a9640e5cf0625ac: No bootstrap required, opened a new log
I20260806 21:15:42.149336 11935 tablet_bootstrap.cc:492] T 16a88a37ca60484f881842676f08620d P f3f46e16e5cc440bb077a4c9200b5b40: No bootstrap required, opened a new log
I20260806 21:15:42.149336 11934 tablet_bootstrap.cc:492] T 16a88a37ca60484f881842676f08620d P 3a7430b190894319a7158be1bd398cbb: No bootstrap required, opened a new log
I20260806 21:15:42.149550 11934 ts_tablet_manager.cc:1397] T 16a88a37ca60484f881842676f08620d P 3a7430b190894319a7158be1bd398cbb: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 21:15:42.149550 11935 ts_tablet_manager.cc:1397] T 16a88a37ca60484f881842676f08620d P f3f46e16e5cc440bb077a4c9200b5b40: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 21:15:42.149600 11933 ts_tablet_manager.cc:1397] T 16a88a37ca60484f881842676f08620d P b3c0a17350a74bff8a9640e5cf0625ac: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 21:15:42.152711 11933 raft_consensus.cc:348] T 16a88a37ca60484f881842676f08620d P b3c0a17350a74bff8a9640e5cf0625ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3f46e16e5cc440bb077a4c9200b5b40" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 33135 } } peers { permanent_uuid: "b3c0a17350a74bff8a9640e5cf0625ac" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 40709 } } peers { permanent_uuid: "3a7430b190894319a7158be1bd398cbb" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 45847 } }
I20260806 21:15:42.152992 11933 raft_consensus.cc:374] T 16a88a37ca60484f881842676f08620d P b3c0a17350a74bff8a9640e5cf0625ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:15:42.153043 11933 raft_consensus.cc:729] T 16a88a37ca60484f881842676f08620d P b3c0a17350a74bff8a9640e5cf0625ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3c0a17350a74bff8a9640e5cf0625ac, State: Initialized, Role: FOLLOWER
I20260806 21:15:42.153525 11933 consensus_queue.cc:260] T 16a88a37ca60484f881842676f08620d P b3c0a17350a74bff8a9640e5cf0625ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3f46e16e5cc440bb077a4c9200b5b40" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 33135 } } peers { permanent_uuid: "b3c0a17350a74bff8a9640e5cf0625ac" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 40709 } } peers { permanent_uuid: "3a7430b190894319a7158be1bd398cbb" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 45847 } }
I20260806 21:15:42.153515 11934 raft_consensus.cc:348] T 16a88a37ca60484f881842676f08620d P 3a7430b190894319a7158be1bd398cbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3f46e16e5cc440bb077a4c9200b5b40" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 33135 } } peers { permanent_uuid: "b3c0a17350a74bff8a9640e5cf0625ac" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 40709 } } peers { permanent_uuid: "3a7430b190894319a7158be1bd398cbb" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 45847 } }
I20260806 21:15:42.153782 11934 raft_consensus.cc:374] T 16a88a37ca60484f881842676f08620d P 3a7430b190894319a7158be1bd398cbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:15:42.153848 11934 raft_consensus.cc:729] T 16a88a37ca60484f881842676f08620d P 3a7430b190894319a7158be1bd398cbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a7430b190894319a7158be1bd398cbb, State: Initialized, Role: FOLLOWER
I20260806 21:15:42.153882 11935 raft_consensus.cc:348] T 16a88a37ca60484f881842676f08620d P f3f46e16e5cc440bb077a4c9200b5b40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3f46e16e5cc440bb077a4c9200b5b40" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 33135 } } peers { permanent_uuid: "b3c0a17350a74bff8a9640e5cf0625ac" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 40709 } } peers { permanent_uuid: "3a7430b190894319a7158be1bd398cbb" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 45847 } }
I20260806 21:15:42.154098 11935 raft_consensus.cc:374] T 16a88a37ca60484f881842676f08620d P f3f46e16e5cc440bb077a4c9200b5b40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:15:42.154152 11935 raft_consensus.cc:729] T 16a88a37ca60484f881842676f08620d P f3f46e16e5cc440bb077a4c9200b5b40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3f46e16e5cc440bb077a4c9200b5b40, State: Initialized, Role: FOLLOWER
I20260806 21:15:42.154335 11934 consensus_queue.cc:260] T 16a88a37ca60484f881842676f08620d P 3a7430b190894319a7158be1bd398cbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3f46e16e5cc440bb077a4c9200b5b40" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 33135 } } peers { permanent_uuid: "b3c0a17350a74bff8a9640e5cf0625ac" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 40709 } } peers { permanent_uuid: "3a7430b190894319a7158be1bd398cbb" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 45847 } }
I20260806 21:15:42.154587 11935 consensus_queue.cc:260] T 16a88a37ca60484f881842676f08620d P f3f46e16e5cc440bb077a4c9200b5b40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3f46e16e5cc440bb077a4c9200b5b40" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 33135 } } peers { permanent_uuid: "b3c0a17350a74bff8a9640e5cf0625ac" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 40709 } } peers { permanent_uuid: "3a7430b190894319a7158be1bd398cbb" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 45847 } }
I20260806 21:15:42.154680 11933 ts_tablet_manager.cc:1428] T 16a88a37ca60484f881842676f08620d P b3c0a17350a74bff8a9640e5cf0625ac: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 21:15:42.155087 11909 heartbeater.cc:503] Master 127.16.178.62:35091 was elected leader, sending a full tablet report...
I20260806 21:15:42.155468 11935 ts_tablet_manager.cc:1428] T 16a88a37ca60484f881842676f08620d P f3f46e16e5cc440bb077a4c9200b5b40: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 21:15:42.155797 11517 heartbeater.cc:503] Master 127.16.178.62:35091 was elected leader, sending a full tablet report...
I20260806 21:15:42.156476 11934 ts_tablet_manager.cc:1428] T 16a88a37ca60484f881842676f08620d P 3a7430b190894319a7158be1bd398cbb: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260806 21:15:42.156821 11720 heartbeater.cc:503] Master 127.16.178.62:35091 was elected leader, sending a full tablet report...
I20260806 21:15:42.230912 11939 raft_consensus.cc:482] T 16a88a37ca60484f881842676f08620d P b3c0a17350a74bff8a9640e5cf0625ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:15:42.231142 11939 raft_consensus.cc:504] T 16a88a37ca60484f881842676f08620d P b3c0a17350a74bff8a9640e5cf0625ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3f46e16e5cc440bb077a4c9200b5b40" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 33135 } } peers { permanent_uuid: "b3c0a17350a74bff8a9640e5cf0625ac" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 40709 } } peers { permanent_uuid: "3a7430b190894319a7158be1bd398cbb" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 45847 } }
I20260806 21:15:42.232429 11939 leader_election.cc:288] T 16a88a37ca60484f881842676f08620d P b3c0a17350a74bff8a9640e5cf0625ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3f46e16e5cc440bb077a4c9200b5b40 (127.16.178.1:33135), 3a7430b190894319a7158be1bd398cbb (127.16.178.2:45847)
I20260806 21:15:42.236444 11912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:42.232725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:33135 (local address 127.16.178.3:38359)
0806 21:15:42.232886 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:42.232914 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:42.233013 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:42.233342 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:42.233345 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:42.233375 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:15:42.233598 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:42.233606 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:42.234427 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:42.234430 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:42.235284 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:42.235295 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:42.235977 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:42.235981 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:42.236144 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:42.236168 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 21:15:42.236236 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 21:15:42.236296 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72,"mutex_wait_us":1}
I20260806 21:15:42.236498 11925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:42.232849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:38359 (local address 127.16.178.1:33135)
0806 21:15:42.232986 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:42.232990 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:42.233010 (+ 20us) server_negotiation.cc:407] Connection header received
0806 21:15:42.233125 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:42.233128 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:42.233183 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:42.233277 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:15:42.233744 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:42.234320 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:42.235426 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:42.235864 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:42.235962 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:42.236009 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:42.236267 (+ 258us) server_negotiation.cc:299] Negotiation successful
0806 21:15:42.236346 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 21:15:42.236510 11930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:42.232960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:47603 (local address 127.16.178.2:45847)
0806 21:15:42.233080 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:42.233084 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:42.233329 (+ 245us) server_negotiation.cc:407] Connection header received
0806 21:15:42.233396 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:42.233399 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:42.233560 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:42.233657 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:15:42.233981 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:42.234484 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:42.235548 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:42.235972 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:42.236110 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:42.236162 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:42.236337 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 21:15:42.236406 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 21:15:42.236675 11944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:42.232750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.2:45847 (local address 127.16.178.3:47603)
0806 21:15:42.233207 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:42.233220 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:42.233317 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:42.233706 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:42.233709 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:42.233730 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:15:42.233848 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:42.233854 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:42.234618 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:42.234620 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:42.235421 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:42.235431 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:42.236116 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:42.236120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:42.236209 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:42.236233 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 21:15:42.236311 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 21:15:42.236372 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":141,"threads_started":1}
I20260806 21:15:42.237171 11447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16a88a37ca60484f881842676f08620d" candidate_uuid: "b3c0a17350a74bff8a9640e5cf0625ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3f46e16e5cc440bb077a4c9200b5b40" is_pre_election: true
I20260806 21:15:42.237290 11650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16a88a37ca60484f881842676f08620d" candidate_uuid: "b3c0a17350a74bff8a9640e5cf0625ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a7430b190894319a7158be1bd398cbb" is_pre_election: true
I20260806 21:15:42.237478 11447 raft_consensus.cc:2454] T 16a88a37ca60484f881842676f08620d P f3f46e16e5cc440bb077a4c9200b5b40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3c0a17350a74bff8a9640e5cf0625ac in term 0.
I20260806 21:15:42.237596 11650 raft_consensus.cc:2454] T 16a88a37ca60484f881842676f08620d P 3a7430b190894319a7158be1bd398cbb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3c0a17350a74bff8a9640e5cf0625ac in term 0.
I20260806 21:15:42.237952 11756 leader_election.cc:302] T 16a88a37ca60484f881842676f08620d P b3c0a17350a74bff8a9640e5cf0625ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3c0a17350a74bff8a9640e5cf0625ac, f3f46e16e5cc440bb077a4c9200b5b40; no voters:
I20260806 21:15:42.238221 11939 raft_consensus.cc:2781] T 16a88a37ca60484f881842676f08620d P b3c0a17350a74bff8a9640e5cf0625ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:15:42.238303 11939 raft_consensus.cc:482] T 16a88a37ca60484f881842676f08620d P b3c0a17350a74bff8a9640e5cf0625ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:15:42.238354 11939 raft_consensus.cc:3037] T 16a88a37ca60484f881842676f08620d P b3c0a17350a74bff8a9640e5cf0625ac [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:15:42.239405 11939 raft_consensus.cc:504] T 16a88a37ca60484f881842676f08620d P b3c0a17350a74bff8a9640e5cf0625ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3f46e16e5cc440bb077a4c9200b5b40" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 33135 } } peers { permanent_uuid: "b3c0a17350a74bff8a9640e5cf0625ac" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 40709 } } peers { permanent_uuid: "3a7430b190894319a7158be1bd398cbb" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 45847 } }
I20260806 21:15:42.239751 11939 leader_election.cc:288] T 16a88a37ca60484f881842676f08620d P b3c0a17350a74bff8a9640e5cf0625ac [CANDIDATE]: Term 1 election: Requested vote from peers f3f46e16e5cc440bb077a4c9200b5b40 (127.16.178.1:33135), 3a7430b190894319a7158be1bd398cbb (127.16.178.2:45847)
I20260806 21:15:42.240140 11447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16a88a37ca60484f881842676f08620d" candidate_uuid: "b3c0a17350a74bff8a9640e5cf0625ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3f46e16e5cc440bb077a4c9200b5b40"
I20260806 21:15:42.240267 11447 raft_consensus.cc:3037] T 16a88a37ca60484f881842676f08620d P f3f46e16e5cc440bb077a4c9200b5b40 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:15:42.240274 11650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16a88a37ca60484f881842676f08620d" candidate_uuid: "b3c0a17350a74bff8a9640e5cf0625ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a7430b190894319a7158be1bd398cbb"
I20260806 21:15:42.240393 11650 raft_consensus.cc:3037] T 16a88a37ca60484f881842676f08620d P 3a7430b190894319a7158be1bd398cbb [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:15:42.241483 11650 raft_consensus.cc:2454] T 16a88a37ca60484f881842676f08620d P 3a7430b190894319a7158be1bd398cbb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3c0a17350a74bff8a9640e5cf0625ac in term 1.
I20260806 21:15:42.241475 11447 raft_consensus.cc:2454] T 16a88a37ca60484f881842676f08620d P f3f46e16e5cc440bb077a4c9200b5b40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3c0a17350a74bff8a9640e5cf0625ac in term 1.
I20260806 21:15:42.241824 11757 leader_election.cc:302] T 16a88a37ca60484f881842676f08620d P b3c0a17350a74bff8a9640e5cf0625ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3a7430b190894319a7158be1bd398cbb, b3c0a17350a74bff8a9640e5cf0625ac, f3f46e16e5cc440bb077a4c9200b5b40; no voters:
I20260806 21:15:42.242061 11939 raft_consensus.cc:2781] T 16a88a37ca60484f881842676f08620d P b3c0a17350a74bff8a9640e5cf0625ac [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:15:42.242278 11939 raft_consensus.cc:686] T 16a88a37ca60484f881842676f08620d P b3c0a17350a74bff8a9640e5cf0625ac [term 1 LEADER]: Becoming Leader. State: Replica: b3c0a17350a74bff8a9640e5cf0625ac, State: Running, Role: LEADER
I20260806 21:15:42.242653 11939 consensus_queue.cc:237] T 16a88a37ca60484f881842676f08620d P b3c0a17350a74bff8a9640e5cf0625ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3f46e16e5cc440bb077a4c9200b5b40" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 33135 } } peers { permanent_uuid: "b3c0a17350a74bff8a9640e5cf0625ac" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 40709 } } peers { permanent_uuid: "3a7430b190894319a7158be1bd398cbb" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 45847 } }
I20260806 21:15:42.246011 11248 catalog_manager.cc:5495] T 16a88a37ca60484f881842676f08620d P b3c0a17350a74bff8a9640e5cf0625ac reported cstate change: term changed from 0 to 1, leader changed from <none> to b3c0a17350a74bff8a9640e5cf0625ac (127.16.178.3). New cstate: current_term: 1 leader_uuid: "b3c0a17350a74bff8a9640e5cf0625ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3f46e16e5cc440bb077a4c9200b5b40" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 33135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3c0a17350a74bff8a9640e5cf0625ac" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 40709 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a7430b190894319a7158be1bd398cbb" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 45847 } health_report { overall_health: UNKNOWN } } }
I20260806 21:15:42.286761 11928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:42.282487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36088 (local address 127.16.178.3:40709)
0806 21:15:42.282645 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:42.282650 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:42.282683 (+ 33us) server_negotiation.cc:407] Connection header received
0806 21:15:42.282778 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:42.282780 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:42.282840 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:42.282938 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:15:42.283252 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:42.283866 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:42.284955 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:42.285198 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:42.285313 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:42.285772 (+ 459us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:15:42.286280 (+ 508us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:15:42.286407 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:42.286517 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 21:15:42.286595 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 21:15:42.294910 11650 raft_consensus.cc:1261] T 16a88a37ca60484f881842676f08620d P 3a7430b190894319a7158be1bd398cbb [term 1 FOLLOWER]: Refusing update from remote peer b3c0a17350a74bff8a9640e5cf0625ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:15:42.294910 11447 raft_consensus.cc:1261] T 16a88a37ca60484f881842676f08620d P f3f46e16e5cc440bb077a4c9200b5b40 [term 1 FOLLOWER]: Refusing update from remote peer b3c0a17350a74bff8a9640e5cf0625ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:15:42.295933 11947 consensus_queue.cc:1035] T 16a88a37ca60484f881842676f08620d P b3c0a17350a74bff8a9640e5cf0625ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3f46e16e5cc440bb077a4c9200b5b40" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 33135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:15:42.296187 11939 consensus_queue.cc:1035] T 16a88a37ca60484f881842676f08620d P b3c0a17350a74bff8a9640e5cf0625ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a7430b190894319a7158be1bd398cbb" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 45847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:15:42.314931 11250 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58574:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:15:42.315148 11250 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58574:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:15:42.318024 11250 catalog_manager.cc:5782] T 00000000000000000000000000000000 P de0a6385cb3142059c77e7a033717094: Sending DeleteTablet for 3 replicas of tablet 16a88a37ca60484f881842676f08620d
I20260806 21:15:42.318979 11422 tablet_service.cc:1514] Processing DeleteTablet for tablet 16a88a37ca60484f881842676f08620d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:15:42 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:36492
I20260806 21:15:42.319087 11619 tablet_service.cc:1514] Processing DeleteTablet for tablet 16a88a37ca60484f881842676f08620d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:15:42 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:58780
I20260806 21:15:42.319927 11814 tablet_service.cc:1514] Processing DeleteTablet for tablet 16a88a37ca60484f881842676f08620d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:15:42 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:36074
I20260806 21:15:42.320295 11966 tablet_replica.cc:330] T 16a88a37ca60484f881842676f08620d P 3a7430b190894319a7158be1bd398cbb: stopping tablet replica
I20260806 21:15:42.320294 11955 mvcc.cc:205] Tried to move back new op lower bound from 7315664659629129728 to 7315664659428630528. Current Snapshot: MvccSnapshot[applied={T|T < 7315664659629129728}]
I20260806 21:15:42.320642 11966 raft_consensus.cc:2229] T 16a88a37ca60484f881842676f08620d P 3a7430b190894319a7158be1bd398cbb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:15:42.320959 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 11345
I20260806 21:15:42.320976 11966 raft_consensus.cc:2258] T 16a88a37ca60484f881842676f08620d P 3a7430b190894319a7158be1bd398cbb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:15:42.335180 11966 ts_tablet_manager.cc:1905] T 16a88a37ca60484f881842676f08620d P 3a7430b190894319a7158be1bd398cbb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:15:42.338660 11966 ts_tablet_manager.cc:1918] T 16a88a37ca60484f881842676f08620d P 3a7430b190894319a7158be1bd398cbb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:15:42.338816 11966 log.cc:1196] T 16a88a37ca60484f881842676f08620d P 3a7430b190894319a7158be1bd398cbb: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-1/wal/wals/16a88a37ca60484f881842676f08620d
I20260806 21:15:42.339144 11966 ts_tablet_manager.cc:1939] T 16a88a37ca60484f881842676f08620d P 3a7430b190894319a7158be1bd398cbb: Deleting consensus metadata
I20260806 21:15:42.340360 11971 tablet_replica.cc:330] T 16a88a37ca60484f881842676f08620d P b3c0a17350a74bff8a9640e5cf0625ac: stopping tablet replica
I20260806 21:15:42.340776 11232 catalog_manager.cc:4841] TS 3a7430b190894319a7158be1bd398cbb (127.16.178.2:45847): tablet 16a88a37ca60484f881842676f08620d (table test-table [id=fb5e494943c44cc5b150554b7c75070e]) successfully deleted
I20260806 21:15:42.340880 11971 raft_consensus.cc:2229] T 16a88a37ca60484f881842676f08620d P b3c0a17350a74bff8a9640e5cf0625ac [term 1 LEADER]: Raft consensus shutting down.
I20260806 21:15:42.341403 11971 raft_consensus.cc:2258] T 16a88a37ca60484f881842676f08620d P b3c0a17350a74bff8a9640e5cf0625ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:15:42.343101 11971 ts_tablet_manager.cc:1905] T 16a88a37ca60484f881842676f08620d P b3c0a17350a74bff8a9640e5cf0625ac: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 21:15:42.344694 11229 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.178.1:33135 (error 108)
I20260806 21:15:42.344877 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 11524
I20260806 21:15:42.348176 11922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:42.347816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:33135 (local address 127.0.0.1:36500)
0806 21:15:42.347963 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:42.348045 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.1:33135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 21:15:42.348500 11229 catalog_manager.cc:4584] TS f3f46e16e5cc440bb077a4c9200b5b40 (127.16.178.1:33135): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 16a88a37ca60484f881842676f08620d: Network error: Client connection negotiation failed: client connection to 127.16.178.1:33135: connect: Connection refused (error 111)
I20260806 21:15:42.349612 11971 ts_tablet_manager.cc:1918] T 16a88a37ca60484f881842676f08620d P b3c0a17350a74bff8a9640e5cf0625ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:15:42.349735 11971 log.cc:1196] T 16a88a37ca60484f881842676f08620d P b3c0a17350a74bff8a9640e5cf0625ac: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786050686797304-17096-0/minicluster-data/ts-2/wal/wals/16a88a37ca60484f881842676f08620d
I20260806 21:15:42.350091 11971 ts_tablet_manager.cc:1939] T 16a88a37ca60484f881842676f08620d P b3c0a17350a74bff8a9640e5cf0625ac: Deleting consensus metadata
I20260806 21:15:42.351686 11234 catalog_manager.cc:4841] TS b3c0a17350a74bff8a9640e5cf0625ac (127.16.178.3:40709): tablet 16a88a37ca60484f881842676f08620d (table test-table [id=fb5e494943c44cc5b150554b7c75070e]) successfully deleted
I20260806 21:15:42.363060 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 11727
I20260806 21:15:42.378626 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 11208
2026-08-06T21:15:42Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2591 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-06T21:15:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:15:42Z Disabled control of system clock
I20260806 21:15:42.419760 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:41181
--webserver_interface=127.16.178.62
--webserver_port=0
--builtin_ntp_servers=127.16.178.20:43621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:41181
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 21:15:42.535436 11977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:15:42.535728 11977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:15:42.535764 11977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:15:42.539808 11977 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:15:42.539876 11977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:15:42.539891 11977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:15:42.539907 11977 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:15:42.539921 11977 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:15:42.544759 11977 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:43621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:41181
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:41181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.178.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:15:42.545867 11977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:15:42.547219 11977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:15:42.555759 11985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:42.555802 11990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:42.555769 11977 server_base.cc:1034] running on GCE node
W20260806 21:15:42.555795 11988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:42.556718 11977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:15:42.557884 11977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:15:42.559054 11977 hybrid_clock.cc:648] HybridClock initialized: now 1786050942559040 us; error 33 us; skew 500 ppm
I20260806 21:15:42.560993 11977 webserver.cc:460] Webserver started at http://127.16.178.62:35719/ using document root <none> and password file <none>
I20260806 21:15:42.561595 11977 fs_manager.cc:359] Metadata directory not provided
I20260806 21:15:42.561667 11977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:15:42.561879 11977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:15:42.564174 11977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "fd2bac18d6d849d5bd3d1dc4510d8a93"
format_stamp: "Formatted at 2026-08-06 21:15:42 on dist-test-slave-lgr5"
server_key: "00f3300c186afdcdb94354e101818d23"
server_key_iv: "7369ea3603cde90d9be287edebbcc050"
server_key_version: "encryptionkey@0"
I20260806 21:15:42.564796 11977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "fd2bac18d6d849d5bd3d1dc4510d8a93"
format_stamp: "Formatted at 2026-08-06 21:15:42 on dist-test-slave-lgr5"
server_key: "00f3300c186afdcdb94354e101818d23"
server_key_iv: "7369ea3603cde90d9be287edebbcc050"
server_key_version: "encryptionkey@0"
I20260806 21:15:42.569172 11977 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260806 21:15:42.572607 11996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:42.574288 11977 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 21:15:42.574482 11977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "fd2bac18d6d849d5bd3d1dc4510d8a93"
format_stamp: "Formatted at 2026-08-06 21:15:42 on dist-test-slave-lgr5"
server_key: "00f3300c186afdcdb94354e101818d23"
server_key_iv: "7369ea3603cde90d9be287edebbcc050"
server_key_version: "encryptionkey@0"
I20260806 21:15:42.574661 11977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:15:42.618162 11977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:15:42.618945 11977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:15:42.619141 11977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:15:42.626121 11977 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.62:41181
I20260806 21:15:42.626188 12068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.62:41181 every 8 connection(s)
I20260806 21:15:42.627173 11977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:15:42.630513 12069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:15:42.636384 12069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd2bac18d6d849d5bd3d1dc4510d8a93: Bootstrap starting.
I20260806 21:15:42.636710 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 11977
I20260806 21:15:42.636827 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:15:42.637101 17096 external_mini_cluster.cc:1428] Setting key 2ad91a263240d7e793697ecb2baba709
I20260806 21:15:42.639043 12069 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd2bac18d6d849d5bd3d1dc4510d8a93: Neither blocks nor log segments found. Creating new log.
I20260806 21:15:42.639791 12069 log.cc:824] T 00000000000000000000000000000000 P fd2bac18d6d849d5bd3d1dc4510d8a93: Log is configured to *not* fsync() on all Append() calls
I20260806 21:15:42.641904 12069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd2bac18d6d849d5bd3d1dc4510d8a93: No bootstrap required, opened a new log
I20260806 21:15:42.646124 12069 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd2bac18d6d849d5bd3d1dc4510d8a93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd2bac18d6d849d5bd3d1dc4510d8a93" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 41181 } }
I20260806 21:15:42.646378 12069 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd2bac18d6d849d5bd3d1dc4510d8a93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:15:42.646436 12069 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd2bac18d6d849d5bd3d1dc4510d8a93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd2bac18d6d849d5bd3d1dc4510d8a93, State: Initialized, Role: FOLLOWER
I20260806 21:15:42.646756 12073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:42.638567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48748 (local address 127.16.178.62:41181)
0806 21:15:42.639232 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:42.639247 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:42.639286 (+ 39us) server_negotiation.cc:407] Connection header received
0806 21:15:42.639946 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:42.639969 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:42.640430 (+ 461us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:42.640727 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:15:42.641078 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:42.642062 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:42.643272 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:42.643786 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:42.643911 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:42.644834 (+ 923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:15:42.644839 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:15:42.644853 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:15:42.645247 (+ 394us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:15:42.645305 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:15:42.645325 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:15:42.645406 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:42.645562 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 21:15:42.645851 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":125,"threads_started":1}
I20260806 21:15:42.646869 12069 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd2bac18d6d849d5bd3d1dc4510d8a93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd2bac18d6d849d5bd3d1dc4510d8a93" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 41181 } }
I20260806 21:15:42.647035 12069 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd2bac18d6d849d5bd3d1dc4510d8a93 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:15:42.647095 12069 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd2bac18d6d849d5bd3d1dc4510d8a93 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:15:42.647182 12069 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd2bac18d6d849d5bd3d1dc4510d8a93 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:15:42.648269 12069 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd2bac18d6d849d5bd3d1dc4510d8a93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd2bac18d6d849d5bd3d1dc4510d8a93" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 41181 } }
I20260806 21:15:42.648590 12069 leader_election.cc:302] T 00000000000000000000000000000000 P fd2bac18d6d849d5bd3d1dc4510d8a93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd2bac18d6d849d5bd3d1dc4510d8a93; no voters:
I20260806 21:15:42.648904 12069 leader_election.cc:288] T 00000000000000000000000000000000 P fd2bac18d6d849d5bd3d1dc4510d8a93 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:15:42.649000 12077 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd2bac18d6d849d5bd3d1dc4510d8a93 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:15:42.649220 12077 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd2bac18d6d849d5bd3d1dc4510d8a93 [term 1 LEADER]: Becoming Leader. State: Replica: fd2bac18d6d849d5bd3d1dc4510d8a93, State: Running, Role: LEADER
I20260806 21:15:42.649539 12077 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd2bac18d6d849d5bd3d1dc4510d8a93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd2bac18d6d849d5bd3d1dc4510d8a93" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 41181 } }
I20260806 21:15:42.650105 12069 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd2bac18d6d849d5bd3d1dc4510d8a93 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:15:42.651150 12080 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd2bac18d6d849d5bd3d1dc4510d8a93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd2bac18d6d849d5bd3d1dc4510d8a93. Latest consensus state: current_term: 1 leader_uuid: "fd2bac18d6d849d5bd3d1dc4510d8a93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd2bac18d6d849d5bd3d1dc4510d8a93" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 41181 } } }
I20260806 21:15:42.651284 12080 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd2bac18d6d849d5bd3d1dc4510d8a93 [sys.catalog]: This master's current role is: LEADER
I20260806 21:15:42.651540 12079 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd2bac18d6d849d5bd3d1dc4510d8a93 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd2bac18d6d849d5bd3d1dc4510d8a93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd2bac18d6d849d5bd3d1dc4510d8a93" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 41181 } } }
I20260806 21:15:42.651635 12079 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd2bac18d6d849d5bd3d1dc4510d8a93 [sys.catalog]: This master's current role is: LEADER
I20260806 21:15:42.651762 12090 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:15:42.655095 12090 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:15:42.661415 12090 catalog_manager.cc:1347] Generated new cluster ID: 671e1f2e8d0a42a5a58951abe0f8e0ee
I20260806 21:15:42.661527 12090 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:15:42.678707 12090 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:15:42.680264 12090 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:15:42.691187 12090 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd2bac18d6d849d5bd3d1dc4510d8a93: Generated new TSK 0
I20260806 21:15:42.692099 12090 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:15:42.707115 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.1:0
--local_ip_for_outbound_sockets=127.16.178.1
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:41181
--builtin_ntp_servers=127.16.178.20:43621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 21:15:42.831777 12103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:15:42.832123 12103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:15:42.832186 12103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:15:42.836385 12103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:15:42.836448 12103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:15:42.836513 12103 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.1
I20260806 21:15:42.841487 12103 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:43621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:41181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12103
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.1
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:15:42.842650 12103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:15:42.844154 12103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:15:42.854633 12110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:42.854635 12108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:42.854633 12112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:42.855050 12103 server_base.cc:1034] running on GCE node
I20260806 21:15:42.855536 12103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:15:42.856318 12103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:15:42.857497 12103 hybrid_clock.cc:648] HybridClock initialized: now 1786050942857469 us; error 46 us; skew 500 ppm
I20260806 21:15:42.859547 12103 webserver.cc:460] Webserver started at http://127.16.178.1:36971/ using document root <none> and password file <none>
I20260806 21:15:42.860212 12103 fs_manager.cc:359] Metadata directory not provided
I20260806 21:15:42.860296 12103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:15:42.860524 12103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:15:42.862756 12103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-0/data/instance:
uuid: "233b7c4d0aba437cb7d0ee0d53496359"
format_stamp: "Formatted at 2026-08-06 21:15:42 on dist-test-slave-lgr5"
server_key: "5af202ec65eb4377388a2f6d5fd2a3fe"
server_key_iv: "c943a819ca4e9b8ed4eb91f7c8dc1c73"
server_key_version: "encryptionkey@0"
I20260806 21:15:42.863356 12103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance:
uuid: "233b7c4d0aba437cb7d0ee0d53496359"
format_stamp: "Formatted at 2026-08-06 21:15:42 on dist-test-slave-lgr5"
server_key: "5af202ec65eb4377388a2f6d5fd2a3fe"
server_key_iv: "c943a819ca4e9b8ed4eb91f7c8dc1c73"
server_key_version: "encryptionkey@0"
I20260806 21:15:42.867835 12103 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260806 21:15:42.870719 12124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:42.872206 12103 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:15:42.872341 12103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-0/wal
uuid: "233b7c4d0aba437cb7d0ee0d53496359"
format_stamp: "Formatted at 2026-08-06 21:15:42 on dist-test-slave-lgr5"
server_key: "5af202ec65eb4377388a2f6d5fd2a3fe"
server_key_iv: "c943a819ca4e9b8ed4eb91f7c8dc1c73"
server_key_version: "encryptionkey@0"
I20260806 21:15:42.872467 12103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:15:42.898662 12103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:15:42.899462 12103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:15:42.899665 12103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:15:42.900345 12103 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:15:42.901397 12103 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:15:42.901449 12103 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:42.901508 12103 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:15:42.901544 12103 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:42.911056 12103 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.1:35295
I20260806 21:15:42.911084 12296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.1:35295 every 8 connection(s)
I20260806 21:15:42.912124 12103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
I20260806 21:15:42.916432 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 12103
I20260806 21:15:42.916566 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance
I20260806 21:15:42.916857 17096 external_mini_cluster.cc:1428] Setting key 70d828c64fc1695d12a0054775f889d4
I20260806 21:15:42.919037 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.2:0
--local_ip_for_outbound_sockets=127.16.178.2
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:41181
--builtin_ntp_servers=127.16.178.20:43621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 21:15:42.923180 12073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:42.914215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:40591 (local address 127.16.178.62:41181)
0806 21:15:42.914360 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:42.914365 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:42.915621 (+ 1256us) server_negotiation.cc:407] Connection header received
0806 21:15:42.917403 (+ 1782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:42.917407 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:42.917485 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:42.917575 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:15:42.918809 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:42.919409 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:42.920819 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:42.921057 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:42.921178 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:42.922640 (+ 1462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:15:42.922644 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:15:42.922647 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:15:42.922668 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:15:42.922693 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:15:42.922703 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:15:42.922764 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:42.922966 (+ 202us) server_negotiation.cc:299] Negotiation successful
0806 21:15:42.923038 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 21:15:42.924788 12299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:42.914638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:41181 (local address 127.16.178.1:40591)
0806 21:15:42.915425 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:42.915481 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:42.917100 (+ 1619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:42.917831 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:42.917841 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:42.917952 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:15:42.918631 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:42.918651 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:42.919570 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:42.919573 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:42.920660 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:42.920676 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:42.921269 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:42.921274 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:42.921465 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:42.922194 (+ 729us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 21:15:42.922197 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:15:42.922270 (+ 73us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 21:15:42.922388 (+ 118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:15:42.922790 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:15:42.922803 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:15:42.922810 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 21:15:42.922969 (+ 159us) client_negotiation.cc:240] Negotiation successful
0806 21:15:42.923377 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":146,"threads_started":1}
I20260806 21:15:42.926661 12297 heartbeater.cc:347] Connected to a master server at 127.16.178.62:41181
I20260806 21:15:42.927066 12297 heartbeater.cc:464] Registering TS with master...
I20260806 21:15:42.927983 12297 heartbeater.cc:511] Master 127.16.178.62:41181 requested a full tablet report, sending...
I20260806 21:15:42.930485 12020 ts_manager.cc:194] Registered new tserver with Master: 233b7c4d0aba437cb7d0ee0d53496359 (127.16.178.1:35295)
I20260806 21:15:42.931852 12020 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.178.1:40591
W20260806 21:15:43.044342 12300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:15:43.044653 12300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:15:43.044694 12300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:15:43.048933 12300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:15:43.049014 12300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:15:43.049098 12300 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.2
I20260806 21:15:43.054554 12300 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:43621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:41181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.2
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:15:43.055788 12300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:15:43.057329 12300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:15:43.068014 12311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:43.068049 12306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:43.068049 12308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:43.068266 12300 server_base.cc:1034] running on GCE node
I20260806 21:15:43.068732 12300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:15:43.069487 12300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:15:43.070673 12300 hybrid_clock.cc:648] HybridClock initialized: now 1786050943070640 us; error 44 us; skew 500 ppm
I20260806 21:15:43.072805 12300 webserver.cc:460] Webserver started at http://127.16.178.2:45331/ using document root <none> and password file <none>
I20260806 21:15:43.073486 12300 fs_manager.cc:359] Metadata directory not provided
I20260806 21:15:43.073558 12300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:15:43.073848 12300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:15:43.076457 12300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-1/data/instance:
uuid: "5213d407f6af47ebac471422e1b8f634"
format_stamp: "Formatted at 2026-08-06 21:15:43 on dist-test-slave-lgr5"
server_key: "54544a598b94fba1e2057a8057135580"
server_key_iv: "af040f111e20ebe12966f36efd26474c"
server_key_version: "encryptionkey@0"
I20260806 21:15:43.077176 12300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance:
uuid: "5213d407f6af47ebac471422e1b8f634"
format_stamp: "Formatted at 2026-08-06 21:15:43 on dist-test-slave-lgr5"
server_key: "54544a598b94fba1e2057a8057135580"
server_key_iv: "af040f111e20ebe12966f36efd26474c"
server_key_version: "encryptionkey@0"
I20260806 21:15:43.081856 12300 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:15:43.085034 12317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:43.086624 12300 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 21:15:43.086786 12300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-1/wal
uuid: "5213d407f6af47ebac471422e1b8f634"
format_stamp: "Formatted at 2026-08-06 21:15:43 on dist-test-slave-lgr5"
server_key: "54544a598b94fba1e2057a8057135580"
server_key_iv: "af040f111e20ebe12966f36efd26474c"
server_key_version: "encryptionkey@0"
I20260806 21:15:43.086925 12300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:15:43.105221 12300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:15:43.105990 12300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:15:43.106160 12300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:15:43.106760 12300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:15:43.107779 12300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:15:43.107841 12300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:43.107903 12300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:15:43.107928 12300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:43.117604 12300 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.2:35197
I20260806 21:15:43.117640 12467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.2:35197 every 8 connection(s)
I20260806 21:15:43.118793 12300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
I20260806 21:15:43.130657 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 12300
I20260806 21:15:43.130815 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance
I20260806 21:15:43.131210 17096 external_mini_cluster.cc:1428] Setting key 7e7e6073a1bed18bc82f50aa7d397faa
I20260806 21:15:43.133963 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.3:0
--local_ip_for_outbound_sockets=127.16.178.3
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:41181
--builtin_ntp_servers=127.16.178.20:43621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 21:15:43.134716 12073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:43.120907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:41923 (local address 127.16.178.62:41181)
0806 21:15:43.121127 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:43.121133 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:43.122656 (+ 1523us) server_negotiation.cc:407] Connection header received
0806 21:15:43.124246 (+ 1590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:43.124249 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:43.124313 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:43.124389 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:15:43.125630 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:43.126184 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:43.131894 (+ 5710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:43.132301 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:43.132439 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:43.134111 (+ 1672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:15:43.134115 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:15:43.134118 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:15:43.134134 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:15:43.134158 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:15:43.134167 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:15:43.134240 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:43.134496 (+ 256us) server_negotiation.cc:299] Negotiation successful
0806 21:15:43.134569 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 21:15:43.136036 12470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:43.121464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:41181 (local address 127.16.178.2:41923)
0806 21:15:43.122391 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:43.122504 (+ 113us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:43.123994 (+ 1490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:43.124635 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:43.124642 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:43.124735 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:15:43.125453 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:43.125476 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:43.126328 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:43.126332 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:43.127996 (+ 1664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:43.128020 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:43.132559 (+ 4539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:43.132565 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:43.132871 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:43.133663 (+ 792us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 21:15:43.133665 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:15:43.133739 (+ 74us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 21:15:43.133869 (+ 130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:15:43.134258 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:15:43.134272 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:15:43.134280 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 21:15:43.134466 (+ 186us) client_negotiation.cc:240] Negotiation successful
0806 21:15:43.134860 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":166,"threads_started":1}
I20260806 21:15:43.137746 12468 heartbeater.cc:347] Connected to a master server at 127.16.178.62:41181
I20260806 21:15:43.138069 12468 heartbeater.cc:464] Registering TS with master...
I20260806 21:15:43.138760 12468 heartbeater.cc:511] Master 127.16.178.62:41181 requested a full tablet report, sending...
I20260806 21:15:43.140196 12020 ts_manager.cc:194] Registered new tserver with Master: 5213d407f6af47ebac471422e1b8f634 (127.16.178.2:35197)
I20260806 21:15:43.140870 12020 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.178.2:41923
W20260806 21:15:43.255187 12472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:15:43.255530 12472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:15:43.255589 12472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:15:43.259857 12472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:15:43.259958 12472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:15:43.260064 12472 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.3
I20260806 21:15:43.265689 12472 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:43621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:41181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.3
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:15:43.266984 12472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:15:43.268617 12472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:15:43.280027 12480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:43.280045 12483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:43.280112 12481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:43.280416 12472 server_base.cc:1034] running on GCE node
I20260806 21:15:43.280985 12472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:15:43.281812 12472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:15:43.283020 12472 hybrid_clock.cc:648] HybridClock initialized: now 1786050943282992 us; error 39 us; skew 500 ppm
I20260806 21:15:43.285634 12472 webserver.cc:460] Webserver started at http://127.16.178.3:34407/ using document root <none> and password file <none>
I20260806 21:15:43.286403 12472 fs_manager.cc:359] Metadata directory not provided
I20260806 21:15:43.286481 12472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:15:43.286813 12472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:15:43.289358 12472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-2/data/instance:
uuid: "3aff0fa033bc46788d52d121414b92aa"
format_stamp: "Formatted at 2026-08-06 21:15:43 on dist-test-slave-lgr5"
server_key: "3b043a0d86fb432bf71a9c99df000832"
server_key_iv: "3391e5830aa70b021440e3f22bbe56c0"
server_key_version: "encryptionkey@0"
I20260806 21:15:43.289999 12472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance:
uuid: "3aff0fa033bc46788d52d121414b92aa"
format_stamp: "Formatted at 2026-08-06 21:15:43 on dist-test-slave-lgr5"
server_key: "3b043a0d86fb432bf71a9c99df000832"
server_key_iv: "3391e5830aa70b021440e3f22bbe56c0"
server_key_version: "encryptionkey@0"
I20260806 21:15:43.295063 12472 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 21:15:43.298326 12490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:43.299808 12472 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:15:43.299959 12472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-2/wal
uuid: "3aff0fa033bc46788d52d121414b92aa"
format_stamp: "Formatted at 2026-08-06 21:15:43 on dist-test-slave-lgr5"
server_key: "3b043a0d86fb432bf71a9c99df000832"
server_key_iv: "3391e5830aa70b021440e3f22bbe56c0"
server_key_version: "encryptionkey@0"
I20260806 21:15:43.300161 12472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:15:43.335322 12472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:15:43.336156 12472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:15:43.336367 12472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:15:43.337014 12472 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:15:43.338059 12472 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:15:43.338109 12472 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:43.338164 12472 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:15:43.338195 12472 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:43.347249 12472 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.3:33171
I20260806 21:15:43.347296 12652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.3:33171 every 8 connection(s)
I20260806 21:15:43.348320 12472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
I20260806 21:15:43.353542 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 12472
I20260806 21:15:43.353645 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance
I20260806 21:15:43.353947 17096 external_mini_cluster.cc:1428] Setting key 112e1027acd16901dd30b6b3f52a2218
I20260806 21:15:43.358776 12073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:43.350293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:54899 (local address 127.16.178.62:41181)
0806 21:15:43.350488 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:43.350494 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:43.351688 (+ 1194us) server_negotiation.cc:407] Connection header received
0806 21:15:43.353318 (+ 1630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:43.353322 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:43.353395 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:43.353472 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:15:43.354625 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:43.355359 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:43.356799 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:43.357041 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:43.357136 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:43.358289 (+ 1153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:15:43.358293 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:15:43.358296 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:15:43.358312 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:15:43.358332 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:15:43.358340 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:15:43.358400 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:43.358551 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 21:15:43.358621 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 21:15:43.359720 12657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:43.350754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:41181 (local address 127.16.178.3:54899)
0806 21:15:43.351486 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:43.351579 (+ 93us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:43.353071 (+ 1492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:43.353693 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:43.353703 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:43.353800 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:15:43.354471 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:43.354493 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:43.355609 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:43.355613 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:43.356678 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:43.356689 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:43.357205 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:43.357210 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:43.357375 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:43.358005 (+ 630us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 21:15:43.358007 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:15:43.358053 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 21:15:43.358130 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:15:43.358410 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:15:43.358420 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:15:43.358426 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 21:15:43.358542 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 21:15:43.358865 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":143,"threads_started":1}
I20260806 21:15:43.360980 12654 heartbeater.cc:347] Connected to a master server at 127.16.178.62:41181
I20260806 21:15:43.361232 12654 heartbeater.cc:464] Registering TS with master...
I20260806 21:15:43.361807 12654 heartbeater.cc:511] Master 127.16.178.62:41181 requested a full tablet report, sending...
I20260806 21:15:43.362929 12020 ts_manager.cc:194] Registered new tserver with Master: 3aff0fa033bc46788d52d121414b92aa (127.16.178.3:33171)
I20260806 21:15:43.363546 12020 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.178.3:54899
I20260806 21:15:43.368590 17096 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 21:15:43.374763 12073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:43.370750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48752 (local address 127.16.178.62:41181)
0806 21:15:43.370889 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:43.370893 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:43.371051 (+ 158us) server_negotiation.cc:407] Connection header received
0806 21:15:43.371188 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:43.371191 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:43.371240 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:43.371307 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:15:43.371630 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:43.372223 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:43.373101 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:43.373332 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:43.373421 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:43.374303 (+ 882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:15:43.374306 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:15:43.374308 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:15:43.374322 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:15:43.374342 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:15:43.374349 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:15:43.374411 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:43.374535 (+ 124us) server_negotiation.cc:299] Negotiation successful
0806 21:15:43.374607 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 21:15:43.378528 12020 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48752:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:15:43.381966 12020 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:15:43.399375 12671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:43.392655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.3:33171 (local address 127.0.0.1:57562)
0806 21:15:43.393837 (+ 1182us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:43.393896 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:43.394132 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:43.394994 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:43.394999 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:43.395036 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:15:43.395168 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:43.395173 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:43.396606 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:43.396610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:43.397854 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:43.397864 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:43.398749 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:43.398752 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:43.398878 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:43.398923 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 21:15:43.399061 (+ 138us) client_negotiation.cc:240] Negotiation successful
0806 21:15:43.399206 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":984,"thread_start_us":146,"threads_started":1}
I20260806 21:15:43.399413 12674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:43.392956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57562 (local address 127.16.178.3:33171)
0806 21:15:43.393526 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:43.393542 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:43.394003 (+ 461us) server_negotiation.cc:407] Connection header received
0806 21:15:43.394337 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:43.394343 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:43.394587 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:43.394736 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:15:43.395293 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:43.396475 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:43.397982 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:43.398616 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:43.398729 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:43.398957 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:43.399118 (+ 161us) server_negotiation.cc:299] Negotiation successful
0806 21:15:43.399235 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":162,"threads_started":1}
I20260806 21:15:43.400012 12672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:43.392746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.2:35197 (local address 127.0.0.1:60364)
0806 21:15:43.394381 (+ 1635us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:43.394395 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:43.394574 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:43.395269 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:43.395272 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:43.395301 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:15:43.395460 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:43.395465 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:43.396818 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:43.396820 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:43.398355 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:43.398368 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:43.399619 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:43.399623 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:43.399714 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:43.399733 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 21:15:43.399798 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 21:15:43.399868 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1573,"mutex_wait_us":21,"spinlock_wait_cycles":14336,"thread_start_us":144,"threads_started":1}
I20260806 21:15:43.400143 12677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:43.393650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60364 (local address 127.16.178.2:35197)
0806 21:15:43.394095 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:43.394108 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:43.394487 (+ 379us) server_negotiation.cc:407] Connection header received
0806 21:15:43.394730 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:43.394735 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:43.394939 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:43.395089 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:15:43.395606 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:43.396704 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:43.398506 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:43.399254 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:43.399356 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:43.399549 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:43.399851 (+ 302us) server_negotiation.cc:299] Negotiation successful
0806 21:15:43.399958 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":118,"threads_started":1}
I20260806 21:15:43.400604 12678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:43.393807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43068 (local address 127.16.178.1:35295)
0806 21:15:43.394274 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:43.394280 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:43.394487 (+ 207us) server_negotiation.cc:407] Connection header received
0806 21:15:43.394729 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:43.394735 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:43.394926 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:43.395090 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:15:43.396172 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:43.397120 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:43.398535 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:43.399282 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:43.399382 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:43.399688 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:43.400293 (+ 605us) server_negotiation.cc:299] Negotiation successful
0806 21:15:43.400399 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":151,"threads_started":1}
I20260806 21:15:43.400197 12670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:43.392866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:35295 (local address 127.0.0.1:43068)
0806 21:15:43.394382 (+ 1516us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:43.394395 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:43.394574 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:43.395269 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:43.395272 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:43.395299 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:15:43.395458 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:43.395465 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:43.397250 (+ 1785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:43.397253 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:43.398396 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:43.398404 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:43.399753 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:43.399757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:43.399863 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:43.399888 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 21:15:43.399957 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 21:15:43.400032 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1471,"mutex_wait_us":54,"spinlock_wait_cycles":7680,"thread_start_us":107,"threads_started":1}
I20260806 21:15:43.403481 12201 tablet_service.cc:1467] Processing CreateTablet for tablet 0bbbbfe87136498b98c4eae13710dff5 (DEFAULT_TABLE table=test-table [id=202a6f1ec334428e8f92ec3d9df3a62e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:15:43.403350 12380 tablet_service.cc:1467] Processing CreateTablet for tablet 0bbbbfe87136498b98c4eae13710dff5 (DEFAULT_TABLE table=test-table [id=202a6f1ec334428e8f92ec3d9df3a62e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:15:43.403350 12560 tablet_service.cc:1467] Processing CreateTablet for tablet 0bbbbfe87136498b98c4eae13710dff5 (DEFAULT_TABLE table=test-table [id=202a6f1ec334428e8f92ec3d9df3a62e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:15:43.404548 12380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bbbbfe87136498b98c4eae13710dff5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:15:43.404548 12560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bbbbfe87136498b98c4eae13710dff5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:15:43.404549 12201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bbbbfe87136498b98c4eae13710dff5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:15:43.410862 12679 tablet_bootstrap.cc:492] T 0bbbbfe87136498b98c4eae13710dff5 P 233b7c4d0aba437cb7d0ee0d53496359: Bootstrap starting.
I20260806 21:15:43.412500 12680 tablet_bootstrap.cc:492] T 0bbbbfe87136498b98c4eae13710dff5 P 5213d407f6af47ebac471422e1b8f634: Bootstrap starting.
I20260806 21:15:43.412563 12682 tablet_bootstrap.cc:492] T 0bbbbfe87136498b98c4eae13710dff5 P 3aff0fa033bc46788d52d121414b92aa: Bootstrap starting.
I20260806 21:15:43.413432 12679 tablet_bootstrap.cc:654] T 0bbbbfe87136498b98c4eae13710dff5 P 233b7c4d0aba437cb7d0ee0d53496359: Neither blocks nor log segments found. Creating new log.
I20260806 21:15:43.414368 12679 log.cc:824] T 0bbbbfe87136498b98c4eae13710dff5 P 233b7c4d0aba437cb7d0ee0d53496359: Log is configured to *not* fsync() on all Append() calls
I20260806 21:15:43.414619 12680 tablet_bootstrap.cc:654] T 0bbbbfe87136498b98c4eae13710dff5 P 5213d407f6af47ebac471422e1b8f634: Neither blocks nor log segments found. Creating new log.
I20260806 21:15:43.414701 12682 tablet_bootstrap.cc:654] T 0bbbbfe87136498b98c4eae13710dff5 P 3aff0fa033bc46788d52d121414b92aa: Neither blocks nor log segments found. Creating new log.
I20260806 21:15:43.415324 12680 log.cc:824] T 0bbbbfe87136498b98c4eae13710dff5 P 5213d407f6af47ebac471422e1b8f634: Log is configured to *not* fsync() on all Append() calls
I20260806 21:15:43.415432 12682 log.cc:824] T 0bbbbfe87136498b98c4eae13710dff5 P 3aff0fa033bc46788d52d121414b92aa: Log is configured to *not* fsync() on all Append() calls
I20260806 21:15:43.417321 12679 tablet_bootstrap.cc:492] T 0bbbbfe87136498b98c4eae13710dff5 P 233b7c4d0aba437cb7d0ee0d53496359: No bootstrap required, opened a new log
I20260806 21:15:43.417440 12680 tablet_bootstrap.cc:492] T 0bbbbfe87136498b98c4eae13710dff5 P 5213d407f6af47ebac471422e1b8f634: No bootstrap required, opened a new log
I20260806 21:15:43.417440 12682 tablet_bootstrap.cc:492] T 0bbbbfe87136498b98c4eae13710dff5 P 3aff0fa033bc46788d52d121414b92aa: No bootstrap required, opened a new log
I20260806 21:15:43.417537 12679 ts_tablet_manager.cc:1397] T 0bbbbfe87136498b98c4eae13710dff5 P 233b7c4d0aba437cb7d0ee0d53496359: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 21:15:43.417634 12682 ts_tablet_manager.cc:1397] T 0bbbbfe87136498b98c4eae13710dff5 P 3aff0fa033bc46788d52d121414b92aa: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 21:15:43.417634 12680 ts_tablet_manager.cc:1397] T 0bbbbfe87136498b98c4eae13710dff5 P 5213d407f6af47ebac471422e1b8f634: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 21:15:43.421339 12679 raft_consensus.cc:348] T 0bbbbfe87136498b98c4eae13710dff5 P 233b7c4d0aba437cb7d0ee0d53496359 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3aff0fa033bc46788d52d121414b92aa" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33171 } } peers { permanent_uuid: "5213d407f6af47ebac471422e1b8f634" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 35197 } } peers { permanent_uuid: "233b7c4d0aba437cb7d0ee0d53496359" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35295 } }
I20260806 21:15:43.421598 12679 raft_consensus.cc:374] T 0bbbbfe87136498b98c4eae13710dff5 P 233b7c4d0aba437cb7d0ee0d53496359 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:15:43.421672 12679 raft_consensus.cc:729] T 0bbbbfe87136498b98c4eae13710dff5 P 233b7c4d0aba437cb7d0ee0d53496359 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233b7c4d0aba437cb7d0ee0d53496359, State: Initialized, Role: FOLLOWER
I20260806 21:15:43.421918 12682 raft_consensus.cc:348] T 0bbbbfe87136498b98c4eae13710dff5 P 3aff0fa033bc46788d52d121414b92aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3aff0fa033bc46788d52d121414b92aa" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33171 } } peers { permanent_uuid: "5213d407f6af47ebac471422e1b8f634" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 35197 } } peers { permanent_uuid: "233b7c4d0aba437cb7d0ee0d53496359" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35295 } }
I20260806 21:15:43.421918 12680 raft_consensus.cc:348] T 0bbbbfe87136498b98c4eae13710dff5 P 5213d407f6af47ebac471422e1b8f634 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3aff0fa033bc46788d52d121414b92aa" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33171 } } peers { permanent_uuid: "5213d407f6af47ebac471422e1b8f634" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 35197 } } peers { permanent_uuid: "233b7c4d0aba437cb7d0ee0d53496359" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35295 } }
I20260806 21:15:43.422178 12682 raft_consensus.cc:374] T 0bbbbfe87136498b98c4eae13710dff5 P 3aff0fa033bc46788d52d121414b92aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:15:43.422178 12680 raft_consensus.cc:374] T 0bbbbfe87136498b98c4eae13710dff5 P 5213d407f6af47ebac471422e1b8f634 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:15:43.422130 12679 consensus_queue.cc:260] T 0bbbbfe87136498b98c4eae13710dff5 P 233b7c4d0aba437cb7d0ee0d53496359 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3aff0fa033bc46788d52d121414b92aa" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33171 } } peers { permanent_uuid: "5213d407f6af47ebac471422e1b8f634" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 35197 } } peers { permanent_uuid: "233b7c4d0aba437cb7d0ee0d53496359" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35295 } }
I20260806 21:15:43.422250 12680 raft_consensus.cc:729] T 0bbbbfe87136498b98c4eae13710dff5 P 5213d407f6af47ebac471422e1b8f634 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5213d407f6af47ebac471422e1b8f634, State: Initialized, Role: FOLLOWER
I20260806 21:15:43.422276 12682 raft_consensus.cc:729] T 0bbbbfe87136498b98c4eae13710dff5 P 3aff0fa033bc46788d52d121414b92aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3aff0fa033bc46788d52d121414b92aa, State: Initialized, Role: FOLLOWER
I20260806 21:15:43.422732 12680 consensus_queue.cc:260] T 0bbbbfe87136498b98c4eae13710dff5 P 5213d407f6af47ebac471422e1b8f634 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3aff0fa033bc46788d52d121414b92aa" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33171 } } peers { permanent_uuid: "5213d407f6af47ebac471422e1b8f634" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 35197 } } peers { permanent_uuid: "233b7c4d0aba437cb7d0ee0d53496359" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35295 } }
I20260806 21:15:43.422866 12297 heartbeater.cc:503] Master 127.16.178.62:41181 was elected leader, sending a full tablet report...
I20260806 21:15:43.423200 12679 ts_tablet_manager.cc:1428] T 0bbbbfe87136498b98c4eae13710dff5 P 233b7c4d0aba437cb7d0ee0d53496359: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
I20260806 21:15:43.423480 12468 heartbeater.cc:503] Master 127.16.178.62:41181 was elected leader, sending a full tablet report...
I20260806 21:15:43.424033 12682 consensus_queue.cc:260] T 0bbbbfe87136498b98c4eae13710dff5 P 3aff0fa033bc46788d52d121414b92aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3aff0fa033bc46788d52d121414b92aa" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33171 } } peers { permanent_uuid: "5213d407f6af47ebac471422e1b8f634" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 35197 } } peers { permanent_uuid: "233b7c4d0aba437cb7d0ee0d53496359" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35295 } }
I20260806 21:15:43.424610 12680 ts_tablet_manager.cc:1428] T 0bbbbfe87136498b98c4eae13710dff5 P 5213d407f6af47ebac471422e1b8f634: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260806 21:15:43.425091 12682 ts_tablet_manager.cc:1428] T 0bbbbfe87136498b98c4eae13710dff5 P 3aff0fa033bc46788d52d121414b92aa: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260806 21:15:43.425457 12654 heartbeater.cc:503] Master 127.16.178.62:41181 was elected leader, sending a full tablet report...
I20260806 21:15:43.445964 12689 raft_consensus.cc:482] T 0bbbbfe87136498b98c4eae13710dff5 P 3aff0fa033bc46788d52d121414b92aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:15:43.446213 12689 raft_consensus.cc:504] T 0bbbbfe87136498b98c4eae13710dff5 P 3aff0fa033bc46788d52d121414b92aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3aff0fa033bc46788d52d121414b92aa" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33171 } } peers { permanent_uuid: "5213d407f6af47ebac471422e1b8f634" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 35197 } } peers { permanent_uuid: "233b7c4d0aba437cb7d0ee0d53496359" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35295 } }
I20260806 21:15:43.447535 12689 leader_election.cc:288] T 0bbbbfe87136498b98c4eae13710dff5 P 3aff0fa033bc46788d52d121414b92aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5213d407f6af47ebac471422e1b8f634 (127.16.178.2:35197), 233b7c4d0aba437cb7d0ee0d53496359 (127.16.178.1:35295)
I20260806 21:15:43.452346 12677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:43.447867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:51103 (local address 127.16.178.2:35197)
0806 21:15:43.448001 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:43.448006 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:43.448110 (+ 104us) server_negotiation.cc:407] Connection header received
0806 21:15:43.448243 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:43.448245 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:43.448290 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:43.448363 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:15:43.448629 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:43.449379 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:43.451038 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:43.451671 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:43.451797 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:43.451850 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:43.452130 (+ 280us) server_negotiation.cc:299] Negotiation successful
0806 21:15:43.452205 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 21:15:43.452346 12678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:43.447909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:58217 (local address 127.16.178.1:35295)
0806 21:15:43.448039 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:43.448044 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:43.448359 (+ 315us) server_negotiation.cc:407] Connection header received
0806 21:15:43.448450 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:43.448453 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:43.448499 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:43.448558 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:15:43.448791 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:43.449507 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:43.451038 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:43.451651 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:43.451797 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:43.451850 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:43.452136 (+ 286us) server_negotiation.cc:299] Negotiation successful
0806 21:15:43.452205 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 21:15:43.452368 12657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:43.447773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.2:35197 (local address 127.16.178.3:51103)
0806 21:15:43.448001 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:43.448017 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:43.448137 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:43.448386 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:43.448389 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:43.448403 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:15:43.448518 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:43.448524 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:43.449521 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:43.449524 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:43.450912 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:43.450924 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:43.451785 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:43.451788 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:43.451979 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:43.452011 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 21:15:43.452109 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 21:15:43.452207 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":113}
I20260806 21:15:43.452385 12690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:43.447794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:35295 (local address 127.16.178.3:58217)
0806 21:15:43.448271 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:43.448282 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:43.448367 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:43.448586 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:43.448589 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:43.448605 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:15:43.448701 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:43.448706 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:43.449614 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:43.449617 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:43.450914 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:43.450933 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:43.451781 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:43.451785 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:43.451952 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:43.452005 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 21:15:43.452109 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 21:15:43.452207 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"mutex_wait_us":84,"thread_start_us":137,"threads_started":1}
I20260806 21:15:43.453034 12408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bbbbfe87136498b98c4eae13710dff5" candidate_uuid: "3aff0fa033bc46788d52d121414b92aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5213d407f6af47ebac471422e1b8f634" is_pre_election: true
I20260806 21:15:43.453331 12408 raft_consensus.cc:2454] T 0bbbbfe87136498b98c4eae13710dff5 P 5213d407f6af47ebac471422e1b8f634 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3aff0fa033bc46788d52d121414b92aa in term 0.
I20260806 21:15:43.453354 12225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bbbbfe87136498b98c4eae13710dff5" candidate_uuid: "3aff0fa033bc46788d52d121414b92aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233b7c4d0aba437cb7d0ee0d53496359" is_pre_election: true
I20260806 21:15:43.453639 12225 raft_consensus.cc:2454] T 0bbbbfe87136498b98c4eae13710dff5 P 233b7c4d0aba437cb7d0ee0d53496359 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3aff0fa033bc46788d52d121414b92aa in term 0.
I20260806 21:15:43.453840 12496 leader_election.cc:302] T 0bbbbfe87136498b98c4eae13710dff5 P 3aff0fa033bc46788d52d121414b92aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aff0fa033bc46788d52d121414b92aa, 5213d407f6af47ebac471422e1b8f634; no voters:
I20260806 21:15:43.454124 12689 raft_consensus.cc:2781] T 0bbbbfe87136498b98c4eae13710dff5 P 3aff0fa033bc46788d52d121414b92aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:15:43.454207 12689 raft_consensus.cc:482] T 0bbbbfe87136498b98c4eae13710dff5 P 3aff0fa033bc46788d52d121414b92aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:15:43.454252 12689 raft_consensus.cc:3037] T 0bbbbfe87136498b98c4eae13710dff5 P 3aff0fa033bc46788d52d121414b92aa [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:15:43.455319 12689 raft_consensus.cc:504] T 0bbbbfe87136498b98c4eae13710dff5 P 3aff0fa033bc46788d52d121414b92aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3aff0fa033bc46788d52d121414b92aa" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33171 } } peers { permanent_uuid: "5213d407f6af47ebac471422e1b8f634" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 35197 } } peers { permanent_uuid: "233b7c4d0aba437cb7d0ee0d53496359" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35295 } }
I20260806 21:15:43.455677 12689 leader_election.cc:288] T 0bbbbfe87136498b98c4eae13710dff5 P 3aff0fa033bc46788d52d121414b92aa [CANDIDATE]: Term 1 election: Requested vote from peers 5213d407f6af47ebac471422e1b8f634 (127.16.178.2:35197), 233b7c4d0aba437cb7d0ee0d53496359 (127.16.178.1:35295)
I20260806 21:15:43.456027 12225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bbbbfe87136498b98c4eae13710dff5" candidate_uuid: "3aff0fa033bc46788d52d121414b92aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233b7c4d0aba437cb7d0ee0d53496359"
I20260806 21:15:43.456172 12225 raft_consensus.cc:3037] T 0bbbbfe87136498b98c4eae13710dff5 P 233b7c4d0aba437cb7d0ee0d53496359 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:15:43.457177 12225 raft_consensus.cc:2454] T 0bbbbfe87136498b98c4eae13710dff5 P 233b7c4d0aba437cb7d0ee0d53496359 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3aff0fa033bc46788d52d121414b92aa in term 1.
I20260806 21:15:43.457449 12408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bbbbfe87136498b98c4eae13710dff5" candidate_uuid: "3aff0fa033bc46788d52d121414b92aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5213d407f6af47ebac471422e1b8f634"
I20260806 21:15:43.457504 12492 leader_election.cc:302] T 0bbbbfe87136498b98c4eae13710dff5 P 3aff0fa033bc46788d52d121414b92aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233b7c4d0aba437cb7d0ee0d53496359, 3aff0fa033bc46788d52d121414b92aa; no voters:
I20260806 21:15:43.457590 12408 raft_consensus.cc:3037] T 0bbbbfe87136498b98c4eae13710dff5 P 5213d407f6af47ebac471422e1b8f634 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:15:43.457681 12689 raft_consensus.cc:2781] T 0bbbbfe87136498b98c4eae13710dff5 P 3aff0fa033bc46788d52d121414b92aa [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:15:43.457914 12689 raft_consensus.cc:686] T 0bbbbfe87136498b98c4eae13710dff5 P 3aff0fa033bc46788d52d121414b92aa [term 1 LEADER]: Becoming Leader. State: Replica: 3aff0fa033bc46788d52d121414b92aa, State: Running, Role: LEADER
I20260806 21:15:43.458273 12689 consensus_queue.cc:237] T 0bbbbfe87136498b98c4eae13710dff5 P 3aff0fa033bc46788d52d121414b92aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3aff0fa033bc46788d52d121414b92aa" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33171 } } peers { permanent_uuid: "5213d407f6af47ebac471422e1b8f634" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 35197 } } peers { permanent_uuid: "233b7c4d0aba437cb7d0ee0d53496359" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35295 } }
I20260806 21:15:43.458689 12408 raft_consensus.cc:2454] T 0bbbbfe87136498b98c4eae13710dff5 P 5213d407f6af47ebac471422e1b8f634 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3aff0fa033bc46788d52d121414b92aa in term 1.
I20260806 21:15:43.461884 12016 catalog_manager.cc:5495] T 0bbbbfe87136498b98c4eae13710dff5 P 3aff0fa033bc46788d52d121414b92aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 3aff0fa033bc46788d52d121414b92aa (127.16.178.3). New cstate: current_term: 1 leader_uuid: "3aff0fa033bc46788d52d121414b92aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3aff0fa033bc46788d52d121414b92aa" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 33171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5213d407f6af47ebac471422e1b8f634" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 35197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "233b7c4d0aba437cb7d0ee0d53496359" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35295 } health_report { overall_health: UNKNOWN } } }
I20260806 21:15:43.484987 12674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:43.481001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57574 (local address 127.16.178.3:33171)
0806 21:15:43.481135 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:43.481138 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:43.481151 (+ 13us) server_negotiation.cc:407] Connection header received
0806 21:15:43.481229 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:43.481231 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:43.481291 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:43.481381 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:15:43.481669 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:43.482171 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:43.483172 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:43.483402 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:43.483486 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:43.483885 (+ 399us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:15:43.484545 (+ 660us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:15:43.484652 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:43.484769 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 21:15:43.484832 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 21:15:43.491334 12225 raft_consensus.cc:1261] T 0bbbbfe87136498b98c4eae13710dff5 P 233b7c4d0aba437cb7d0ee0d53496359 [term 1 FOLLOWER]: Refusing update from remote peer 3aff0fa033bc46788d52d121414b92aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:15:43.491333 12408 raft_consensus.cc:1261] T 0bbbbfe87136498b98c4eae13710dff5 P 5213d407f6af47ebac471422e1b8f634 [term 1 FOLLOWER]: Refusing update from remote peer 3aff0fa033bc46788d52d121414b92aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:15:43.492144 12689 consensus_queue.cc:1035] T 0bbbbfe87136498b98c4eae13710dff5 P 3aff0fa033bc46788d52d121414b92aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "233b7c4d0aba437cb7d0ee0d53496359" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 35295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:15:43.492365 12692 consensus_queue.cc:1035] T 0bbbbfe87136498b98c4eae13710dff5 P 3aff0fa033bc46788d52d121414b92aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "5213d407f6af47ebac471422e1b8f634" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 35197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:15:43.502521 12701 mvcc.cc:205] Tried to move back new op lower bound from 7315664664532787200 to 7315664664408203264. Current Snapshot: MvccSnapshot[applied={T|T < 7315664664532787200}]
I20260806 21:15:43.510654 12020 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:15:43.510838 12020 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:15:43.513568 12020 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fd2bac18d6d849d5bd3d1dc4510d8a93: Sending DeleteTablet for 3 replicas of tablet 0bbbbfe87136498b98c4eae13710dff5
I20260806 21:15:43.514609 12380 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bbbbfe87136498b98c4eae13710dff5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:15:43 UTC) from {username='slave'} at 127.0.0.1:60364
I20260806 21:15:43.514613 12201 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bbbbfe87136498b98c4eae13710dff5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:15:43 UTC) from {username='slave'} at 127.0.0.1:43068
I20260806 21:15:43.515267 12711 tablet_replica.cc:330] T 0bbbbfe87136498b98c4eae13710dff5 P 5213d407f6af47ebac471422e1b8f634: stopping tablet replica
I20260806 21:15:43.515276 12560 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bbbbfe87136498b98c4eae13710dff5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:15:43 UTC) from {username='slave'} at 127.0.0.1:57562
I20260806 21:15:43.515481 12712 tablet_replica.cc:330] T 0bbbbfe87136498b98c4eae13710dff5 P 233b7c4d0aba437cb7d0ee0d53496359: stopping tablet replica
I20260806 21:15:43.515764 12711 raft_consensus.cc:2229] T 0bbbbfe87136498b98c4eae13710dff5 P 5213d407f6af47ebac471422e1b8f634 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:15:43.515818 12712 raft_consensus.cc:2229] T 0bbbbfe87136498b98c4eae13710dff5 P 233b7c4d0aba437cb7d0ee0d53496359 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:15:43.516287 12712 raft_consensus.cc:2258] T 0bbbbfe87136498b98c4eae13710dff5 P 233b7c4d0aba437cb7d0ee0d53496359 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:15:43.516287 12711 raft_consensus.cc:2258] T 0bbbbfe87136498b98c4eae13710dff5 P 5213d407f6af47ebac471422e1b8f634 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:15:43.516561 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 12103
I20260806 21:15:43.518472 12711 ts_tablet_manager.cc:1905] T 0bbbbfe87136498b98c4eae13710dff5 P 5213d407f6af47ebac471422e1b8f634: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:15:43.522375 12711 ts_tablet_manager.cc:1918] T 0bbbbfe87136498b98c4eae13710dff5 P 5213d407f6af47ebac471422e1b8f634: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:15:43.522612 12711 log.cc:1196] T 0bbbbfe87136498b98c4eae13710dff5 P 5213d407f6af47ebac471422e1b8f634: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-1/wal/wals/0bbbbfe87136498b98c4eae13710dff5
I20260806 21:15:43.523023 12711 ts_tablet_manager.cc:1939] T 0bbbbfe87136498b98c4eae13710dff5 P 5213d407f6af47ebac471422e1b8f634: Deleting consensus metadata
I20260806 21:15:43.524961 12000 catalog_manager.cc:4841] TS 5213d407f6af47ebac471422e1b8f634 (127.16.178.2:35197): tablet 0bbbbfe87136498b98c4eae13710dff5 (table test-table [id=202a6f1ec334428e8f92ec3d9df3a62e]) successfully deleted
I20260806 21:15:43.529134 12713 tablet_replica.cc:330] T 0bbbbfe87136498b98c4eae13710dff5 P 3aff0fa033bc46788d52d121414b92aa: stopping tablet replica
I20260806 21:15:43.529559 12713 raft_consensus.cc:2229] T 0bbbbfe87136498b98c4eae13710dff5 P 3aff0fa033bc46788d52d121414b92aa [term 1 LEADER]: Raft consensus shutting down.
I20260806 21:15:43.530036 12713 raft_consensus.cc:2258] T 0bbbbfe87136498b98c4eae13710dff5 P 3aff0fa033bc46788d52d121414b92aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:15:43.533016 12713 ts_tablet_manager.cc:1905] T 0bbbbfe87136498b98c4eae13710dff5 P 3aff0fa033bc46788d52d121414b92aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:15:43.536521 12713 ts_tablet_manager.cc:1918] T 0bbbbfe87136498b98c4eae13710dff5 P 3aff0fa033bc46788d52d121414b92aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:15:43.536772 12713 log.cc:1196] T 0bbbbfe87136498b98c4eae13710dff5 P 3aff0fa033bc46788d52d121414b92aa: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786050686797304-17096-0/minicluster-data/ts-2/wal/wals/0bbbbfe87136498b98c4eae13710dff5
I20260806 21:15:43.537137 12713 ts_tablet_manager.cc:1939] T 0bbbbfe87136498b98c4eae13710dff5 P 3aff0fa033bc46788d52d121414b92aa: Deleting consensus metadata
I20260806 21:15:43.539748 11999 catalog_manager.cc:4841] TS 3aff0fa033bc46788d52d121414b92aa (127.16.178.3:33171): tablet 0bbbbfe87136498b98c4eae13710dff5 (table test-table [id=202a6f1ec334428e8f92ec3d9df3a62e]) successfully deleted
W20260806 21:15:43.547971 11997 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.178.1:35295 (error 108)
I20260806 21:15:43.548657 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 12300
I20260806 21:15:43.551928 12671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:43.551483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:35295 (local address 127.0.0.1:43080)
0806 21:15:43.551720 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:43.551816 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.1:35295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
W20260806 21:15:43.552302 11997 catalog_manager.cc:4584] TS 233b7c4d0aba437cb7d0ee0d53496359 (127.16.178.1:35295): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0bbbbfe87136498b98c4eae13710dff5: Network error: Client connection negotiation failed: client connection to 127.16.178.1:35295: connect: Connection refused (error 111)
I20260806 21:15:43.558523 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 12472
I20260806 21:15:43.574121 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 11977
2026-08-06T21:15:43Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1188 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:15:43 dist-test-slave-lgr5 krb5kdc[12717](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:15:43 dist-test-slave-lgr5 krb5kdc[12717](info): set up 2 sockets
Aug 06 21:15:43 dist-test-slave-lgr5 krb5kdc[12717](info): commencing operation
krb5kdc: starting...
W20260806 21:15:44.739012 17096 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Aug 06 21:15:44 dist-test-slave-lgr5 krb5kdc[12717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:15:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:15:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.178.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
I20260806 21:15:44.873381 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:32987
--webserver_interface=127.16.178.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:37089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:32987
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:15:44.992974 12732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:15:44.993295 12732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:15:44.993355 12732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:15:44.997526 12732 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:15:44.997597 12732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:15:44.997624 12732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:15:44.997645 12732 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:15:44.997674 12732 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:15:45.003583 12732 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:37089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:32987
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:32987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.178.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12732
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:15:45.004988 12732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:15:45.006089 12732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:15:45.015321 12744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:45.015398 12739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:45.015321 12740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:45.015595 12732 server_base.cc:1034] running on GCE node
I20260806 21:15:45.016297 12732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:15:45.017457 12732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:15:45.018625 12732 hybrid_clock.cc:648] HybridClock initialized: now 1786050945018612 us; error 27 us; skew 500 ppm
Aug 06 21:15:45 dist-test-slave-lgr5 krb5kdc[12717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050945, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:15:45.025990 12732 init.cc:377] Logged in from keytab as kudu/127.16.178.62@KRBTEST.COM (short username kudu)
I20260806 21:15:45.027081 12732 webserver.cc:460] Webserver started at http://127.16.178.62:33977/ using document root <none> and password file <none>
I20260806 21:15:45.027683 12732 fs_manager.cc:359] Metadata directory not provided
I20260806 21:15:45.027755 12732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:15:45.028000 12732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:15:45.030234 12732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "0f13ffde91aa4442853c38269107a5bc"
format_stamp: "Formatted at 2026-08-06 21:15:45 on dist-test-slave-lgr5"
server_key: "02528e38cb08602519942714786180df"
server_key_iv: "5a6135eecad5e992799c49fb47f2d299"
server_key_version: "encryptionkey@0"
I20260806 21:15:45.030833 12732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "0f13ffde91aa4442853c38269107a5bc"
format_stamp: "Formatted at 2026-08-06 21:15:45 on dist-test-slave-lgr5"
server_key: "02528e38cb08602519942714786180df"
server_key_iv: "5a6135eecad5e992799c49fb47f2d299"
server_key_version: "encryptionkey@0"
I20260806 21:15:45.035262 12732 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 21:15:45.038187 12751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:45.039595 12732 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:15:45.039760 12732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "0f13ffde91aa4442853c38269107a5bc"
format_stamp: "Formatted at 2026-08-06 21:15:45 on dist-test-slave-lgr5"
server_key: "02528e38cb08602519942714786180df"
server_key_iv: "5a6135eecad5e992799c49fb47f2d299"
server_key_version: "encryptionkey@0"
I20260806 21:15:45.039897 12732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:15:45.055081 12732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:15:45.058467 12732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:15:45.058662 12732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:15:45.066085 12732 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.62:32987
I20260806 21:15:45.066228 12823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.62:32987 every 8 connection(s)
I20260806 21:15:45.067134 12732 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:15:45.070226 12824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:15:45.071521 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 12732
I20260806 21:15:45.071676 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:15:45.072029 17096 external_mini_cluster.cc:1428] Setting key 2878a412e1224a0f33be0d3e524baaf5
I20260806 21:15:45.077175 12824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f13ffde91aa4442853c38269107a5bc: Bootstrap starting.
I20260806 21:15:45.079551 12824 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f13ffde91aa4442853c38269107a5bc: Neither blocks nor log segments found. Creating new log.
I20260806 21:15:45.080349 12824 log.cc:824] T 00000000000000000000000000000000 P 0f13ffde91aa4442853c38269107a5bc: Log is configured to *not* fsync() on all Append() calls
Aug 06 21:15:45 dist-test-slave-lgr5 krb5kdc[12717](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:15:45.082641 12824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f13ffde91aa4442853c38269107a5bc: No bootstrap required, opened a new log
I20260806 21:15:45.086701 12832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:45.073793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52822 (local address 127.16.178.62:32987)
0806 21:15:45.074373 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:45.074390 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:45.074430 (+ 40us) server_negotiation.cc:407] Connection header received
0806 21:15:45.075144 (+ 714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:45.075163 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:45.075519 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:45.075850 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:15:45.076748 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:45.077779 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:45.078669 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:45.079145 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:45.079311 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:45.082012 (+ 2701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:15:45.082017 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:15:45.082032 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:15:45.082062 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:15:45.083767 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:45.084307 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:45.084312 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:45.084317 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:45.084432 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:45.084758 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:45.084760 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:45.084761 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:45.085170 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:15:45.085313 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:45.085614 (+ 301us) server_negotiation.cc:299] Negotiation successful
0806 21:15:45.085906 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":171,"threads_started":1}
I20260806 21:15:45.086710 12824 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f13ffde91aa4442853c38269107a5bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f13ffde91aa4442853c38269107a5bc" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 32987 } }
I20260806 21:15:45.087056 12824 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0f13ffde91aa4442853c38269107a5bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:15:45.087128 12824 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f13ffde91aa4442853c38269107a5bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f13ffde91aa4442853c38269107a5bc, State: Initialized, Role: FOLLOWER
I20260806 21:15:45.087553 12824 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f13ffde91aa4442853c38269107a5bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f13ffde91aa4442853c38269107a5bc" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 32987 } }
I20260806 21:15:45.087685 12824 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f13ffde91aa4442853c38269107a5bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:15:45.087750 12824 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f13ffde91aa4442853c38269107a5bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:15:45.087852 12824 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f13ffde91aa4442853c38269107a5bc [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:15:45.088871 12824 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f13ffde91aa4442853c38269107a5bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f13ffde91aa4442853c38269107a5bc" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 32987 } }
I20260806 21:15:45.089214 12824 leader_election.cc:302] T 00000000000000000000000000000000 P 0f13ffde91aa4442853c38269107a5bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f13ffde91aa4442853c38269107a5bc; no voters:
I20260806 21:15:45.089547 12824 leader_election.cc:288] T 00000000000000000000000000000000 P 0f13ffde91aa4442853c38269107a5bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:15:45.089650 12834 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f13ffde91aa4442853c38269107a5bc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:15:45.089870 12834 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f13ffde91aa4442853c38269107a5bc [term 1 LEADER]: Becoming Leader. State: Replica: 0f13ffde91aa4442853c38269107a5bc, State: Running, Role: LEADER
I20260806 21:15:45.090230 12834 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f13ffde91aa4442853c38269107a5bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f13ffde91aa4442853c38269107a5bc" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 32987 } }
I20260806 21:15:45.090782 12824 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f13ffde91aa4442853c38269107a5bc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:15:45.091751 12836 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f13ffde91aa4442853c38269107a5bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f13ffde91aa4442853c38269107a5bc. Latest consensus state: current_term: 1 leader_uuid: "0f13ffde91aa4442853c38269107a5bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f13ffde91aa4442853c38269107a5bc" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 32987 } } }
I20260806 21:15:45.091873 12836 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f13ffde91aa4442853c38269107a5bc [sys.catalog]: This master's current role is: LEADER
I20260806 21:15:45.092347 12835 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f13ffde91aa4442853c38269107a5bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0f13ffde91aa4442853c38269107a5bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f13ffde91aa4442853c38269107a5bc" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 32987 } } }
I20260806 21:15:45.092476 12835 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f13ffde91aa4442853c38269107a5bc [sys.catalog]: This master's current role is: LEADER
I20260806 21:15:45.092768 12844 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:15:45.095638 12844 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:15:45.101132 12844 catalog_manager.cc:1347] Generated new cluster ID: df551bd4eff441ab88ff6c501755d92b
I20260806 21:15:45.101213 12844 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:15:45.111976 12844 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:15:45.113035 12844 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:15:45.123392 12844 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f13ffde91aa4442853c38269107a5bc: Generated new TSK 0
I20260806 21:15:45.124248 12844 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.178.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab.
Entry for principal kudu/127.16.178.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab.
I20260806 21:15:45.157063 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.1:0
--local_ip_for_outbound_sockets=127.16.178.1
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:32987
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab
--principal=kudu/127.16.178.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:37089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:15:45.281984 12859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:15:45.282337 12859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:15:45.282384 12859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:15:45.286845 12859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:15:45.286935 12859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:15:45.287003 12859 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.1
I20260806 21:15:45.292920 12859 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:37089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab
--principal=kudu/127.16.178.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:32987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12859
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.1
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:15:45.294231 12859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:15:45.295380 12859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:15:45.305861 12865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:45.305876 12864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:45.305943 12859 server_base.cc:1034] running on GCE node
W20260806 21:15:45.305876 12867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:45.306627 12859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:15:45.307315 12859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:15:45.308486 12859 hybrid_clock.cc:648] HybridClock initialized: now 1786050945308452 us; error 42 us; skew 500 ppm
Aug 06 21:15:45 dist-test-slave-lgr5 krb5kdc[12717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050945, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:15:45.311719 12859 init.cc:377] Logged in from keytab as kudu/127.16.178.1@KRBTEST.COM (short username kudu)
I20260806 21:15:45.312718 12859 webserver.cc:460] Webserver started at http://127.16.178.1:39267/ using document root <none> and password file <none>
I20260806 21:15:45.313328 12859 fs_manager.cc:359] Metadata directory not provided
I20260806 21:15:45.313390 12859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:15:45.313568 12859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:15:45.315863 12859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-0/data/instance:
uuid: "c25d6a6dc6aa4c7ba369cb2ba1621821"
format_stamp: "Formatted at 2026-08-06 21:15:45 on dist-test-slave-lgr5"
server_key: "56ce4044585b83e3833a016e72fb9c54"
server_key_iv: "4a0e9805ed08e2fc082459c5c0f969f3"
server_key_version: "encryptionkey@0"
I20260806 21:15:45.316591 12859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance:
uuid: "c25d6a6dc6aa4c7ba369cb2ba1621821"
format_stamp: "Formatted at 2026-08-06 21:15:45 on dist-test-slave-lgr5"
server_key: "56ce4044585b83e3833a016e72fb9c54"
server_key_iv: "4a0e9805ed08e2fc082459c5c0f969f3"
server_key_version: "encryptionkey@0"
I20260806 21:15:45.320749 12859 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:15:45.323518 12874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:45.324868 12859 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:15:45.324991 12859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-0/wal
uuid: "c25d6a6dc6aa4c7ba369cb2ba1621821"
format_stamp: "Formatted at 2026-08-06 21:15:45 on dist-test-slave-lgr5"
server_key: "56ce4044585b83e3833a016e72fb9c54"
server_key_iv: "4a0e9805ed08e2fc082459c5c0f969f3"
server_key_version: "encryptionkey@0"
I20260806 21:15:45.325129 12859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:15:45.349062 12859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:15:45.352339 12859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:15:45.352524 12859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:15:45.353211 12859 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:15:45.354131 12859 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:15:45.354182 12859 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:45.354223 12859 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:15:45.354239 12859 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:45.363173 12859 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.1:37951
I20260806 21:15:45.363202 13031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.1:37951 every 8 connection(s)
I20260806 21:15:45.364164 12859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
I20260806 21:15:45.365159 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 12859
I20260806 21:15:45.365303 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance
I20260806 21:15:45.365630 17096 external_mini_cluster.cc:1428] Setting key 7ce46a6e7271a9c9a9102b4458d1b67e
Aug 06 21:15:45 dist-test-slave-lgr5 krb5kdc[12717](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050945, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.178.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.2@KRBTEST.COM" created.
I20260806 21:15:45.380743 12832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:45.366295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:47989 (local address 127.16.178.62:32987)
0806 21:15:45.366438 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:45.366442 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:45.367430 (+ 988us) server_negotiation.cc:407] Connection header received
0806 21:15:45.368802 (+ 1372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:45.368806 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:45.368881 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:45.368958 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:15:45.370261 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:45.370793 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:45.372627 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:45.372890 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:45.373001 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:45.376698 (+ 3697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:15:45.376703 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:15:45.376705 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:15:45.376740 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:15:45.378351 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:45.378994 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:45.378997 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:45.378999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:45.379084 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:45.379396 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:45.379398 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:45.379400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:45.379650 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:15:45.379739 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:45.380454 (+ 715us) server_negotiation.cc:299] Negotiation successful
0806 21:15:45.380582 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 21:15:45.381687 13036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:45.366728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:32987 (local address 127.16.178.1:47989)
0806 21:15:45.367277 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:45.367335 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:45.368596 (+ 1261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:45.369162 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:45.369169 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:45.369564 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:15:45.370099 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:45.370115 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:45.370968 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:45.370972 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:45.372431 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:45.372450 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:45.373120 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:45.373125 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:45.373402 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:45.374138 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:15:45.374164 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:15:45.376374 (+ 2210us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:15:45.378508 (+ 2134us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:15:45.378545 (+ 37us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:15:45.378563 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:15:45.378882 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:15:45.379155 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:15:45.379157 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:15:45.379158 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:15:45.379317 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:15:45.379753 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:15:45.379760 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:15:45.380062 (+ 302us) client_negotiation.cc:769] Sending connection context
0806 21:15:45.380317 (+ 255us) client_negotiation.cc:240] Negotiation successful
0806 21:15:45.380748 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":149,"threads_started":1}
I20260806 21:15:45.383037 13033 heartbeater.cc:347] Connected to a master server at 127.16.178.62:32987
I20260806 21:15:45.383306 13033 heartbeater.cc:464] Registering TS with master...
I20260806 21:15:45.383972 13033 heartbeater.cc:511] Master 127.16.178.62:32987 requested a full tablet report, sending...
I20260806 21:15:45.385653 12776 ts_manager.cc:194] Registered new tserver with Master: c25d6a6dc6aa4c7ba369cb2ba1621821 (127.16.178.1:37951)
I20260806 21:15:45.386816 12776 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.1@KRBTEST.COM'} at 127.16.178.1:47989
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab.
Entry for principal kudu/127.16.178.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab.
I20260806 21:15:45.398734 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.2:0
--local_ip_for_outbound_sockets=127.16.178.2
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:32987
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab
--principal=kudu/127.16.178.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:37089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:15:45.514786 13039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:15:45.515161 13039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:15:45.515233 13039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:15:45.519410 13039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:15:45.519485 13039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:15:45.519548 13039 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.2
I20260806 21:15:45.525107 13039 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:37089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab
--principal=kudu/127.16.178.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:32987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.2
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:15:45.526242 13039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:15:45.527258 13039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 21:15:45.537298 13039 server_base.cc:1034] running on GCE node
W20260806 21:15:45.537254 13044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:45.537254 13050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:45.537256 13046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:45.537940 13039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:15:45.538580 13039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:15:45.539748 13039 hybrid_clock.cc:648] HybridClock initialized: now 1786050945539731 us; error 33 us; skew 500 ppm
Aug 06 21:15:45 dist-test-slave-lgr5 krb5kdc[12717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050945, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:15:45.542428 13039 init.cc:377] Logged in from keytab as kudu/127.16.178.2@KRBTEST.COM (short username kudu)
I20260806 21:15:45.543334 13039 webserver.cc:460] Webserver started at http://127.16.178.2:42223/ using document root <none> and password file <none>
I20260806 21:15:45.543889 13039 fs_manager.cc:359] Metadata directory not provided
I20260806 21:15:45.543955 13039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:15:45.544204 13039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:15:45.546241 13039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-1/data/instance:
uuid: "7af270eb5c214fcb9395f8906459f08a"
format_stamp: "Formatted at 2026-08-06 21:15:45 on dist-test-slave-lgr5"
server_key: "073db99e619c1ca7b9a55a139623ce8e"
server_key_iv: "1e6f8ea8f82626a7ec05baf9aa0a2ea8"
server_key_version: "encryptionkey@0"
I20260806 21:15:45.546783 13039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance:
uuid: "7af270eb5c214fcb9395f8906459f08a"
format_stamp: "Formatted at 2026-08-06 21:15:45 on dist-test-slave-lgr5"
server_key: "073db99e619c1ca7b9a55a139623ce8e"
server_key_iv: "1e6f8ea8f82626a7ec05baf9aa0a2ea8"
server_key_version: "encryptionkey@0"
I20260806 21:15:45.550560 13039 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:15:45.552999 13061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:45.554188 13039 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260806 21:15:45.554298 13039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-1/wal
uuid: "7af270eb5c214fcb9395f8906459f08a"
format_stamp: "Formatted at 2026-08-06 21:15:45 on dist-test-slave-lgr5"
server_key: "073db99e619c1ca7b9a55a139623ce8e"
server_key_iv: "1e6f8ea8f82626a7ec05baf9aa0a2ea8"
server_key_version: "encryptionkey@0"
I20260806 21:15:45.554392 13039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:15:45.580016 13039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:15:45.583506 13039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:15:45.583717 13039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:15:45.584429 13039 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:15:45.585455 13039 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:15:45.585510 13039 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:45.585572 13039 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:15:45.585595 13039 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:45.595002 13039 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.2:33403
I20260806 21:15:45.595049 13221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.2:33403 every 8 connection(s)
I20260806 21:15:45.596053 13039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
I20260806 21:15:45.596719 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 13039
I20260806 21:15:45.596884 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance
I20260806 21:15:45.597218 17096 external_mini_cluster.cc:1428] Setting key 2d1793b44bb6368d938f7039bc09e4a4
Aug 06 21:15:45 dist-test-slave-lgr5 krb5kdc[12717](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050945, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.2@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.178.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.3@KRBTEST.COM" created.
I20260806 21:15:45.612560 12832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:45.598256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:46727 (local address 127.16.178.62:32987)
0806 21:15:45.598428 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:45.598432 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:45.599651 (+ 1219us) server_negotiation.cc:407] Connection header received
0806 21:15:45.601133 (+ 1482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:45.601138 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:45.601203 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:45.601277 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:15:45.603088 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:45.603635 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:45.605566 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:45.605794 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:45.605883 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:45.609027 (+ 3144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:15:45.609032 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:15:45.609034 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:15:45.609065 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:15:45.610438 (+ 1373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:45.610921 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:45.610924 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:45.610925 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:45.610996 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:45.611332 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:45.611335 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:45.611336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:45.611565 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:15:45.611647 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:45.612245 (+ 598us) server_negotiation.cc:299] Negotiation successful
0806 21:15:45.612361 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 21:15:45.613655 13228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:45.598884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:32987 (local address 127.16.178.2:46727)
0806 21:15:45.599469 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:45.599530 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:45.600854 (+ 1324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:45.601504 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:45.601515 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:45.602062 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:15:45.602862 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:45.602886 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:45.603800 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:45.603804 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:45.605390 (+ 1586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:45.605404 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:45.605981 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:45.605987 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:45.606214 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:45.606900 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:15:45.606922 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:15:45.608790 (+ 1868us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:15:45.610552 (+ 1762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:15:45.610558 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:15:45.610574 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:15:45.610835 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:15:45.611061 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:15:45.611063 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:15:45.611065 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:15:45.611194 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:15:45.611650 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:15:45.611656 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:15:45.611918 (+ 262us) client_negotiation.cc:769] Sending connection context
0806 21:15:45.612171 (+ 253us) client_negotiation.cc:240] Negotiation successful
0806 21:15:45.612554 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":171,"threads_started":1}
I20260806 21:15:45.615038 13224 heartbeater.cc:347] Connected to a master server at 127.16.178.62:32987
I20260806 21:15:45.615343 13224 heartbeater.cc:464] Registering TS with master...
I20260806 21:15:45.616008 13224 heartbeater.cc:511] Master 127.16.178.62:32987 requested a full tablet report, sending...
I20260806 21:15:45.617189 12776 ts_manager.cc:194] Registered new tserver with Master: 7af270eb5c214fcb9395f8906459f08a (127.16.178.2:33403)
I20260806 21:15:45.617810 12776 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.2@KRBTEST.COM'} at 127.16.178.2:46727
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab.
Entry for principal kudu/127.16.178.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab.
I20260806 21:15:45.631124 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.3:0
--local_ip_for_outbound_sockets=127.16.178.3
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:32987
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab
--principal=kudu/127.16.178.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:37089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:15:45.750324 13232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:15:45.750599 13232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:15:45.750658 13232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:15:45.754642 13232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:15:45.754702 13232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:15:45.754783 13232 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.3
I20260806 21:15:45.760053 13232 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:37089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab
--principal=kudu/127.16.178.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:32987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13232
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.3
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:15:45.761164 13232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:15:45.762190 13232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:15:45.772073 13241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:45.772150 13239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:45.772279 13232 server_base.cc:1034] running on GCE node
W20260806 21:15:45.772193 13238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:45.772892 13232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:15:45.773645 13232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:15:45.774829 13232 hybrid_clock.cc:648] HybridClock initialized: now 1786050945774803 us; error 42 us; skew 500 ppm
Aug 06 21:15:45 dist-test-slave-lgr5 krb5kdc[12717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050945, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:15:45.777824 13232 init.cc:377] Logged in from keytab as kudu/127.16.178.3@KRBTEST.COM (short username kudu)
I20260806 21:15:45.778824 13232 webserver.cc:460] Webserver started at http://127.16.178.3:37769/ using document root <none> and password file <none>
I20260806 21:15:45.779362 13232 fs_manager.cc:359] Metadata directory not provided
I20260806 21:15:45.779421 13232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:15:45.779605 13232 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:15:45.781720 13232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-2/data/instance:
uuid: "4b4b603b7e7141b8aef75e5b6db45e33"
format_stamp: "Formatted at 2026-08-06 21:15:45 on dist-test-slave-lgr5"
server_key: "25d15e0b4bd581e9b96a33ecad0217af"
server_key_iv: "bd152f0ffabf46e2db47fcc05f1483f8"
server_key_version: "encryptionkey@0"
I20260806 21:15:45.782249 13232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance:
uuid: "4b4b603b7e7141b8aef75e5b6db45e33"
format_stamp: "Formatted at 2026-08-06 21:15:45 on dist-test-slave-lgr5"
server_key: "25d15e0b4bd581e9b96a33ecad0217af"
server_key_iv: "bd152f0ffabf46e2db47fcc05f1483f8"
server_key_version: "encryptionkey@0"
I20260806 21:15:45.786087 13232 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:15:45.788559 13252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:45.789788 13232 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 21:15:45.789901 13232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-2/wal
uuid: "4b4b603b7e7141b8aef75e5b6db45e33"
format_stamp: "Formatted at 2026-08-06 21:15:45 on dist-test-slave-lgr5"
server_key: "25d15e0b4bd581e9b96a33ecad0217af"
server_key_iv: "bd152f0ffabf46e2db47fcc05f1483f8"
server_key_version: "encryptionkey@0"
I20260806 21:15:45.789994 13232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:15:45.814394 13232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:15:45.817539 13232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:15:45.817724 13232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:15:45.818307 13232 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:15:45.819236 13232 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:15:45.819288 13232 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:45.819327 13232 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:15:45.819350 13232 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:45.828541 13232 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.3:35273
I20260806 21:15:45.828609 13416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.3:35273 every 8 connection(s)
I20260806 21:15:45.829810 13232 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
I20260806 21:15:45.839188 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 13232
I20260806 21:15:45.839318 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance
I20260806 21:15:45.839604 17096 external_mini_cluster.cc:1428] Setting key 0ffb742161ffabc3934019c687283d85
Aug 06 21:15:45 dist-test-slave-lgr5 krb5kdc[12717](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050945, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.3@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:15:45.844305 12832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:45.832037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:40503 (local address 127.16.178.62:32987)
0806 21:15:45.832230 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:45.832235 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:45.833353 (+ 1118us) server_negotiation.cc:407] Connection header received
0806 21:15:45.834730 (+ 1377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:45.834734 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:45.834796 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:45.834877 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:15:45.836228 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:45.836875 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:45.838081 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:45.838305 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:45.838394 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:45.840963 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:15:45.840968 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:15:45.840971 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:15:45.841001 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:15:45.842400 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:45.842895 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:45.842898 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:45.842899 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:45.842974 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:45.843246 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:45.843248 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:45.843249 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:45.843420 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:15:45.843490 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:45.844037 (+ 547us) server_negotiation.cc:299] Negotiation successful
0806 21:15:45.844178 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 21:15:45.845187 13419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:45.832561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:32987 (local address 127.16.178.3:40503)
0806 21:15:45.833149 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:45.833203 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:45.834471 (+ 1268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:45.835071 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:45.835078 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:45.835471 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:15:45.836016 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:45.836032 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:45.837014 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:45.837017 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:45.837916 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:45.837927 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:45.838460 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:45.838463 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:45.838620 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:45.839224 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:15:45.839241 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:15:45.840819 (+ 1578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:15:45.842523 (+ 1704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:15:45.842528 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:15:45.842541 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:15:45.842807 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:15:45.843038 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:15:45.843040 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:15:45.843041 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:15:45.843177 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:15:45.843502 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:15:45.843507 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:15:45.843808 (+ 301us) client_negotiation.cc:769] Sending connection context
0806 21:15:45.843965 (+ 157us) client_negotiation.cc:240] Negotiation successful
0806 21:15:45.844290 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":170,"threads_started":1}
I20260806 21:15:45.846295 13417 heartbeater.cc:347] Connected to a master server at 127.16.178.62:32987
I20260806 21:15:45.846529 13417 heartbeater.cc:464] Registering TS with master...
I20260806 21:15:45.847087 13417 heartbeater.cc:511] Master 127.16.178.62:32987 requested a full tablet report, sending...
I20260806 21:15:45.848232 12776 ts_manager.cc:194] Registered new tserver with Master: 4b4b603b7e7141b8aef75e5b6db45e33 (127.16.178.3:35273)
I20260806 21:15:45.848803 12776 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.3@KRBTEST.COM'} at 127.16.178.3:40503
I20260806 21:15:45.854017 17096 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 21:15:45.863937 12832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:45.856175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52826 (local address 127.16.178.62:32987)
0806 21:15:45.856318 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:45.856323 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:45.856500 (+ 177us) server_negotiation.cc:407] Connection header received
0806 21:15:45.856651 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:45.856655 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:45.856705 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:45.856771 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:15:45.857593 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:45.858135 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:45.859027 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:45.859250 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:45.859341 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:45.860962 (+ 1621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:15:45.860967 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:15:45.860970 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:15:45.860999 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:15:45.862092 (+ 1093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:45.862608 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:45.862612 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:45.862614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:45.862694 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:45.863002 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:45.863006 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:45.863008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:45.863240 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:15:45.863379 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:45.863670 (+ 291us) server_negotiation.cc:299] Negotiation successful
0806 21:15:45.863773 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 21:15:45.867506 12776 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52826:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:15:45.870883 12776 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:15:45.886981 13437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:45.880562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.2:33403 (local address 127.0.0.1:55090)
0806 21:15:45.881817 (+ 1255us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:45.881831 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:45.881961 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:45.882795 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:45.882798 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:45.882815 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:15:45.882947 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:45.882955 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:45.883993 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:45.883997 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:45.885309 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:45.885320 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:45.886361 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:45.886365 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:45.886481 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:45.886535 (+ 54us) client_negotiation.cc:769] Sending connection context
0806 21:15:45.886654 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 21:15:45.886806 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1193,"mutex_wait_us":44,"spinlock_wait_cycles":5632,"thread_start_us":137,"threads_started":1}
I20260806 21:15:45.887192 13433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:45.880450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.3:35273 (local address 127.0.0.1:48492)
0806 21:15:45.881355 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:45.881795 (+ 440us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:45.881961 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:45.882880 (+ 919us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:45.882883 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:45.882898 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:15:45.882996 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:45.883002 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:45.884351 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:45.884354 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:45.885592 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:45.885604 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:45.886855 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:45.886859 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:45.886937 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:45.886958 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 21:15:45.887020 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 21:15:45.887081 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":707,"mutex_wait_us":25,"spinlock_wait_cycles":3328,"thread_start_us":92,"threads_started":1}
I20260806 21:15:45.887091 13441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:45.881350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48074 (local address 127.16.178.1:37951)
0806 21:15:45.881911 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:45.881929 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:45.882164 (+ 235us) server_negotiation.cc:407] Connection header received
0806 21:15:45.882249 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:45.882253 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:45.882441 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:45.882610 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:15:45.882957 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:45.884406 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:45.885865 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:45.886448 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:45.886566 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:45.886709 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:45.886862 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 21:15:45.886961 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":145,"threads_started":1}
I20260806 21:15:45.887544 13438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:45.881345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55090 (local address 127.16.178.2:33403)
0806 21:15:45.881699 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:45.881709 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:45.882109 (+ 400us) server_negotiation.cc:407] Connection header received
0806 21:15:45.882202 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:45.882209 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:45.882442 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:45.882610 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:15:45.883088 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:45.883856 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:45.885442 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:45.886221 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:45.886353 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:45.886531 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:45.887178 (+ 647us) server_negotiation.cc:299] Negotiation successful
0806 21:15:45.887298 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":137,"threads_started":1}
I20260806 21:15:45.887684 13435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:45.880611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48492 (local address 127.16.178.3:35273)
0806 21:15:45.882399 (+ 1788us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:45.882416 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:45.882441 (+ 25us) server_negotiation.cc:407] Connection header received
0806 21:15:45.882530 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:45.882536 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:45.882733 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:45.882897 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:15:45.883080 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:45.884199 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:45.885728 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:45.886542 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:45.886642 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:45.886780 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:45.887436 (+ 656us) server_negotiation.cc:299] Negotiation successful
0806 21:15:45.887552 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1663,"thread_start_us":185,"threads_started":1}
I20260806 21:15:45.887053 13434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:45.880790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:37951 (local address 127.0.0.1:48074)
0806 21:15:45.882050 (+ 1260us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:45.882061 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:45.882151 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:45.882614 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:45.882620 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:45.882647 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:15:45.882812 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:45.882821 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:45.884523 (+ 1702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:45.884527 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:45.885748 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:45.885757 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:45.886623 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:45.886627 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:45.886714 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:45.886739 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 21:15:45.886827 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 21:15:45.886902 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1204,"mutex_wait_us":1,"thread_start_us":105,"threads_started":1}
I20260806 21:15:45.890816 12942 tablet_service.cc:1467] Processing CreateTablet for tablet 53a36d144e854b09a60e87ece224f97e (DEFAULT_TABLE table=test-table [id=291792fb3ac649bd8e9ce9951ffe102e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:15:45.890897 13323 tablet_service.cc:1467] Processing CreateTablet for tablet 53a36d144e854b09a60e87ece224f97e (DEFAULT_TABLE table=test-table [id=291792fb3ac649bd8e9ce9951ffe102e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:15:45.890897 13127 tablet_service.cc:1467] Processing CreateTablet for tablet 53a36d144e854b09a60e87ece224f97e (DEFAULT_TABLE table=test-table [id=291792fb3ac649bd8e9ce9951ffe102e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:15:45.892012 12942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53a36d144e854b09a60e87ece224f97e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:15:45.892151 13127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53a36d144e854b09a60e87ece224f97e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:15:45.892151 13323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53a36d144e854b09a60e87ece224f97e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:15:45.899523 13442 tablet_bootstrap.cc:492] T 53a36d144e854b09a60e87ece224f97e P c25d6a6dc6aa4c7ba369cb2ba1621821: Bootstrap starting.
I20260806 21:15:45.900988 13443 tablet_bootstrap.cc:492] T 53a36d144e854b09a60e87ece224f97e P 4b4b603b7e7141b8aef75e5b6db45e33: Bootstrap starting.
I20260806 21:15:45.901140 13444 tablet_bootstrap.cc:492] T 53a36d144e854b09a60e87ece224f97e P 7af270eb5c214fcb9395f8906459f08a: Bootstrap starting.
I20260806 21:15:45.902415 13442 tablet_bootstrap.cc:654] T 53a36d144e854b09a60e87ece224f97e P c25d6a6dc6aa4c7ba369cb2ba1621821: Neither blocks nor log segments found. Creating new log.
I20260806 21:15:45.903245 13443 tablet_bootstrap.cc:654] T 53a36d144e854b09a60e87ece224f97e P 4b4b603b7e7141b8aef75e5b6db45e33: Neither blocks nor log segments found. Creating new log.
I20260806 21:15:45.903254 13442 log.cc:824] T 53a36d144e854b09a60e87ece224f97e P c25d6a6dc6aa4c7ba369cb2ba1621821: Log is configured to *not* fsync() on all Append() calls
I20260806 21:15:45.903699 13444 tablet_bootstrap.cc:654] T 53a36d144e854b09a60e87ece224f97e P 7af270eb5c214fcb9395f8906459f08a: Neither blocks nor log segments found. Creating new log.
I20260806 21:15:45.904552 13444 log.cc:824] T 53a36d144e854b09a60e87ece224f97e P 7af270eb5c214fcb9395f8906459f08a: Log is configured to *not* fsync() on all Append() calls
I20260806 21:15:45.904585 13443 log.cc:824] T 53a36d144e854b09a60e87ece224f97e P 4b4b603b7e7141b8aef75e5b6db45e33: Log is configured to *not* fsync() on all Append() calls
I20260806 21:15:45.906045 13442 tablet_bootstrap.cc:492] T 53a36d144e854b09a60e87ece224f97e P c25d6a6dc6aa4c7ba369cb2ba1621821: No bootstrap required, opened a new log
I20260806 21:15:45.906221 13442 ts_tablet_manager.cc:1397] T 53a36d144e854b09a60e87ece224f97e P c25d6a6dc6aa4c7ba369cb2ba1621821: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 21:15:45.906862 13443 tablet_bootstrap.cc:492] T 53a36d144e854b09a60e87ece224f97e P 4b4b603b7e7141b8aef75e5b6db45e33: No bootstrap required, opened a new log
I20260806 21:15:45.907068 13443 ts_tablet_manager.cc:1397] T 53a36d144e854b09a60e87ece224f97e P 4b4b603b7e7141b8aef75e5b6db45e33: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 21:15:45.907203 13444 tablet_bootstrap.cc:492] T 53a36d144e854b09a60e87ece224f97e P 7af270eb5c214fcb9395f8906459f08a: No bootstrap required, opened a new log
I20260806 21:15:45.907378 13444 ts_tablet_manager.cc:1397] T 53a36d144e854b09a60e87ece224f97e P 7af270eb5c214fcb9395f8906459f08a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 21:15:45.909781 13442 raft_consensus.cc:348] T 53a36d144e854b09a60e87ece224f97e P c25d6a6dc6aa4c7ba369cb2ba1621821 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4b603b7e7141b8aef75e5b6db45e33" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 35273 } } peers { permanent_uuid: "7af270eb5c214fcb9395f8906459f08a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 33403 } } peers { permanent_uuid: "c25d6a6dc6aa4c7ba369cb2ba1621821" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 37951 } }
I20260806 21:15:45.910044 13442 raft_consensus.cc:374] T 53a36d144e854b09a60e87ece224f97e P c25d6a6dc6aa4c7ba369cb2ba1621821 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:15:45.910086 13442 raft_consensus.cc:729] T 53a36d144e854b09a60e87ece224f97e P c25d6a6dc6aa4c7ba369cb2ba1621821 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c25d6a6dc6aa4c7ba369cb2ba1621821, State: Initialized, Role: FOLLOWER
I20260806 21:15:45.910753 13442 consensus_queue.cc:260] T 53a36d144e854b09a60e87ece224f97e P c25d6a6dc6aa4c7ba369cb2ba1621821 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4b603b7e7141b8aef75e5b6db45e33" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 35273 } } peers { permanent_uuid: "7af270eb5c214fcb9395f8906459f08a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 33403 } } peers { permanent_uuid: "c25d6a6dc6aa4c7ba369cb2ba1621821" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 37951 } }
I20260806 21:15:45.911158 13443 raft_consensus.cc:348] T 53a36d144e854b09a60e87ece224f97e P 4b4b603b7e7141b8aef75e5b6db45e33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4b603b7e7141b8aef75e5b6db45e33" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 35273 } } peers { permanent_uuid: "7af270eb5c214fcb9395f8906459f08a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 33403 } } peers { permanent_uuid: "c25d6a6dc6aa4c7ba369cb2ba1621821" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 37951 } }
I20260806 21:15:45.911158 13444 raft_consensus.cc:348] T 53a36d144e854b09a60e87ece224f97e P 7af270eb5c214fcb9395f8906459f08a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4b603b7e7141b8aef75e5b6db45e33" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 35273 } } peers { permanent_uuid: "7af270eb5c214fcb9395f8906459f08a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 33403 } } peers { permanent_uuid: "c25d6a6dc6aa4c7ba369cb2ba1621821" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 37951 } }
I20260806 21:15:45.911481 13444 raft_consensus.cc:374] T 53a36d144e854b09a60e87ece224f97e P 7af270eb5c214fcb9395f8906459f08a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:15:45.911481 13443 raft_consensus.cc:374] T 53a36d144e854b09a60e87ece224f97e P 4b4b603b7e7141b8aef75e5b6db45e33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:15:45.911535 13444 raft_consensus.cc:729] T 53a36d144e854b09a60e87ece224f97e P 7af270eb5c214fcb9395f8906459f08a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7af270eb5c214fcb9395f8906459f08a, State: Initialized, Role: FOLLOWER
I20260806 21:15:45.911546 13443 raft_consensus.cc:729] T 53a36d144e854b09a60e87ece224f97e P 4b4b603b7e7141b8aef75e5b6db45e33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b4b603b7e7141b8aef75e5b6db45e33, State: Initialized, Role: FOLLOWER
I20260806 21:15:45.912027 13442 ts_tablet_manager.cc:1428] T 53a36d144e854b09a60e87ece224f97e P c25d6a6dc6aa4c7ba369cb2ba1621821: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 21:15:45.912062 13443 consensus_queue.cc:260] T 53a36d144e854b09a60e87ece224f97e P 4b4b603b7e7141b8aef75e5b6db45e33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4b603b7e7141b8aef75e5b6db45e33" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 35273 } } peers { permanent_uuid: "7af270eb5c214fcb9395f8906459f08a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 33403 } } peers { permanent_uuid: "c25d6a6dc6aa4c7ba369cb2ba1621821" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 37951 } }
I20260806 21:15:45.912062 13444 consensus_queue.cc:260] T 53a36d144e854b09a60e87ece224f97e P 7af270eb5c214fcb9395f8906459f08a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4b603b7e7141b8aef75e5b6db45e33" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 35273 } } peers { permanent_uuid: "7af270eb5c214fcb9395f8906459f08a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 33403 } } peers { permanent_uuid: "c25d6a6dc6aa4c7ba369cb2ba1621821" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 37951 } }
I20260806 21:15:45.912523 13033 heartbeater.cc:503] Master 127.16.178.62:32987 was elected leader, sending a full tablet report...
I20260806 21:15:45.913226 13444 ts_tablet_manager.cc:1428] T 53a36d144e854b09a60e87ece224f97e P 7af270eb5c214fcb9395f8906459f08a: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260806 21:15:45.913654 13224 heartbeater.cc:503] Master 127.16.178.62:32987 was elected leader, sending a full tablet report...
I20260806 21:15:45.913842 13417 heartbeater.cc:503] Master 127.16.178.62:32987 was elected leader, sending a full tablet report...
I20260806 21:15:45.914270 13443 ts_tablet_manager.cc:1428] T 53a36d144e854b09a60e87ece224f97e P 4b4b603b7e7141b8aef75e5b6db45e33: Time spent starting tablet: real 0.007s user 0.003s sys 0.004s
I20260806 21:15:45.964061 13448 raft_consensus.cc:482] T 53a36d144e854b09a60e87ece224f97e P c25d6a6dc6aa4c7ba369cb2ba1621821 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:15:45.964356 13448 raft_consensus.cc:504] T 53a36d144e854b09a60e87ece224f97e P c25d6a6dc6aa4c7ba369cb2ba1621821 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4b603b7e7141b8aef75e5b6db45e33" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 35273 } } peers { permanent_uuid: "7af270eb5c214fcb9395f8906459f08a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 33403 } } peers { permanent_uuid: "c25d6a6dc6aa4c7ba369cb2ba1621821" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 37951 } }
I20260806 21:15:45.965591 13448 leader_election.cc:288] T 53a36d144e854b09a60e87ece224f97e P c25d6a6dc6aa4c7ba369cb2ba1621821 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b4b603b7e7141b8aef75e5b6db45e33 (127.16.178.3:35273), 7af270eb5c214fcb9395f8906459f08a (127.16.178.2:33403)
I20260806 21:15:45.969832 13438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:45.966029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:56745 (local address 127.16.178.2:33403)
0806 21:15:45.966172 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:45.966176 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:45.966452 (+ 276us) server_negotiation.cc:407] Connection header received
0806 21:15:45.966634 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:45.966638 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:45.966692 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:45.966750 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:15:45.967081 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:45.967636 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:45.968808 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:45.969274 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:45.969382 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:45.969426 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:45.969625 (+ 199us) server_negotiation.cc:299] Negotiation successful
0806 21:15:45.969700 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 21:15:45.969842 13454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:45.965907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.2:33403 (local address 127.16.178.1:56745)
0806 21:15:45.966351 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:45.966364 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:45.966489 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:45.966780 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:45.966783 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:45.966799 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:15:45.966964 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:45.966971 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:45.967751 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:45.967755 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:45.968682 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:45.968693 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:45.969402 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:45.969406 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:45.969500 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:45.969529 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 21:15:45.969601 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 21:15:45.969666 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"mutex_wait_us":5,"thread_start_us":153,"threads_started":1}
I20260806 21:15:45.969904 13435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:45.966041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:37019 (local address 127.16.178.3:35273)
0806 21:15:45.966172 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:45.966177 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:45.966564 (+ 387us) server_negotiation.cc:407] Connection header received
0806 21:15:45.966692 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:45.966694 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:45.966740 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:45.966806 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:15:45.967127 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:45.967633 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:45.968982 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:45.969402 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:45.969527 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:45.969573 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:45.969736 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 21:15:45.969801 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 21:15:45.969981 13452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:45.965907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.3:35273 (local address 127.16.178.1:37019)
0806 21:15:45.966461 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:45.966473 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:45.966585 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:45.966850 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:45.966853 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:45.966868 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:15:45.967014 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:45.967019 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:45.967747 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:45.967750 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:45.968859 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:45.968869 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:45.969520 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:45.969523 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:45.969613 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:45.969638 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 21:15:45.969708 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 21:15:45.969774 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"mutex_wait_us":102,"thread_start_us":114,"threads_started":1}
I20260806 21:15:45.970636 13161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53a36d144e854b09a60e87ece224f97e" candidate_uuid: "c25d6a6dc6aa4c7ba369cb2ba1621821" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7af270eb5c214fcb9395f8906459f08a" is_pre_election: true
I20260806 21:15:45.970636 13350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53a36d144e854b09a60e87ece224f97e" candidate_uuid: "c25d6a6dc6aa4c7ba369cb2ba1621821" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4b603b7e7141b8aef75e5b6db45e33" is_pre_election: true
I20260806 21:15:45.970973 13161 raft_consensus.cc:2454] T 53a36d144e854b09a60e87ece224f97e P 7af270eb5c214fcb9395f8906459f08a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c25d6a6dc6aa4c7ba369cb2ba1621821 in term 0.
I20260806 21:15:45.970973 13350 raft_consensus.cc:2454] T 53a36d144e854b09a60e87ece224f97e P 4b4b603b7e7141b8aef75e5b6db45e33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c25d6a6dc6aa4c7ba369cb2ba1621821 in term 0.
I20260806 21:15:45.971539 12875 leader_election.cc:302] T 53a36d144e854b09a60e87ece224f97e P c25d6a6dc6aa4c7ba369cb2ba1621821 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7af270eb5c214fcb9395f8906459f08a, c25d6a6dc6aa4c7ba369cb2ba1621821; no voters:
I20260806 21:15:45.971882 13448 raft_consensus.cc:2781] T 53a36d144e854b09a60e87ece224f97e P c25d6a6dc6aa4c7ba369cb2ba1621821 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:15:45.972005 13448 raft_consensus.cc:482] T 53a36d144e854b09a60e87ece224f97e P c25d6a6dc6aa4c7ba369cb2ba1621821 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:15:45.972108 13448 raft_consensus.cc:3037] T 53a36d144e854b09a60e87ece224f97e P c25d6a6dc6aa4c7ba369cb2ba1621821 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:15:45.973604 13448 raft_consensus.cc:504] T 53a36d144e854b09a60e87ece224f97e P c25d6a6dc6aa4c7ba369cb2ba1621821 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4b603b7e7141b8aef75e5b6db45e33" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 35273 } } peers { permanent_uuid: "7af270eb5c214fcb9395f8906459f08a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 33403 } } peers { permanent_uuid: "c25d6a6dc6aa4c7ba369cb2ba1621821" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 37951 } }
I20260806 21:15:45.974056 13448 leader_election.cc:288] T 53a36d144e854b09a60e87ece224f97e P c25d6a6dc6aa4c7ba369cb2ba1621821 [CANDIDATE]: Term 1 election: Requested vote from peers 4b4b603b7e7141b8aef75e5b6db45e33 (127.16.178.3:35273), 7af270eb5c214fcb9395f8906459f08a (127.16.178.2:33403)
I20260806 21:15:45.974495 13350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53a36d144e854b09a60e87ece224f97e" candidate_uuid: "c25d6a6dc6aa4c7ba369cb2ba1621821" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b4b603b7e7141b8aef75e5b6db45e33"
I20260806 21:15:45.974576 13161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53a36d144e854b09a60e87ece224f97e" candidate_uuid: "c25d6a6dc6aa4c7ba369cb2ba1621821" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7af270eb5c214fcb9395f8906459f08a"
I20260806 21:15:45.974627 13350 raft_consensus.cc:3037] T 53a36d144e854b09a60e87ece224f97e P 4b4b603b7e7141b8aef75e5b6db45e33 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:15:45.974785 13161 raft_consensus.cc:3037] T 53a36d144e854b09a60e87ece224f97e P 7af270eb5c214fcb9395f8906459f08a [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:15:45.975922 13350 raft_consensus.cc:2454] T 53a36d144e854b09a60e87ece224f97e P 4b4b603b7e7141b8aef75e5b6db45e33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c25d6a6dc6aa4c7ba369cb2ba1621821 in term 1.
I20260806 21:15:45.976322 13161 raft_consensus.cc:2454] T 53a36d144e854b09a60e87ece224f97e P 7af270eb5c214fcb9395f8906459f08a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c25d6a6dc6aa4c7ba369cb2ba1621821 in term 1.
I20260806 21:15:45.976347 12876 leader_election.cc:302] T 53a36d144e854b09a60e87ece224f97e P c25d6a6dc6aa4c7ba369cb2ba1621821 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b4b603b7e7141b8aef75e5b6db45e33, c25d6a6dc6aa4c7ba369cb2ba1621821; no voters:
I20260806 21:15:45.976557 13448 raft_consensus.cc:2781] T 53a36d144e854b09a60e87ece224f97e P c25d6a6dc6aa4c7ba369cb2ba1621821 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:15:45.976908 13448 raft_consensus.cc:686] T 53a36d144e854b09a60e87ece224f97e P c25d6a6dc6aa4c7ba369cb2ba1621821 [term 1 LEADER]: Becoming Leader. State: Replica: c25d6a6dc6aa4c7ba369cb2ba1621821, State: Running, Role: LEADER
I20260806 21:15:45.977370 13448 consensus_queue.cc:237] T 53a36d144e854b09a60e87ece224f97e P c25d6a6dc6aa4c7ba369cb2ba1621821 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4b603b7e7141b8aef75e5b6db45e33" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 35273 } } peers { permanent_uuid: "7af270eb5c214fcb9395f8906459f08a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 33403 } } peers { permanent_uuid: "c25d6a6dc6aa4c7ba369cb2ba1621821" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 37951 } }
I20260806 21:15:45.981290 12776 catalog_manager.cc:5495] T 53a36d144e854b09a60e87ece224f97e P c25d6a6dc6aa4c7ba369cb2ba1621821 reported cstate change: term changed from 0 to 1, leader changed from <none> to c25d6a6dc6aa4c7ba369cb2ba1621821 (127.16.178.1). New cstate: current_term: 1 leader_uuid: "c25d6a6dc6aa4c7ba369cb2ba1621821" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b4b603b7e7141b8aef75e5b6db45e33" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 35273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7af270eb5c214fcb9395f8906459f08a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 33403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c25d6a6dc6aa4c7ba369cb2ba1621821" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 37951 } health_report { overall_health: HEALTHY } } }
I20260806 21:15:46.014104 13441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:46.010203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48086 (local address 127.16.178.1:37951)
0806 21:15:46.010329 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:46.010333 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:46.010345 (+ 12us) server_negotiation.cc:407] Connection header received
0806 21:15:46.010421 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:46.010423 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:46.010480 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:46.010593 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:15:46.010853 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:46.011360 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:46.012234 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:46.012561 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:46.012667 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:46.013022 (+ 355us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:15:46.013540 (+ 518us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:15:46.013678 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:46.013935 (+ 257us) server_negotiation.cc:299] Negotiation successful
0806 21:15:46.013982 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 21:15:46.021143 13161 raft_consensus.cc:1261] T 53a36d144e854b09a60e87ece224f97e P 7af270eb5c214fcb9395f8906459f08a [term 1 FOLLOWER]: Refusing update from remote peer c25d6a6dc6aa4c7ba369cb2ba1621821: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:15:46.021143 13350 raft_consensus.cc:1261] T 53a36d144e854b09a60e87ece224f97e P 4b4b603b7e7141b8aef75e5b6db45e33 [term 1 FOLLOWER]: Refusing update from remote peer c25d6a6dc6aa4c7ba369cb2ba1621821: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:15:46.022059 13456 consensus_queue.cc:1035] T 53a36d144e854b09a60e87ece224f97e P c25d6a6dc6aa4c7ba369cb2ba1621821 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b4b603b7e7141b8aef75e5b6db45e33" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 35273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:15:46.022325 13448 consensus_queue.cc:1035] T 53a36d144e854b09a60e87ece224f97e P c25d6a6dc6aa4c7ba369cb2ba1621821 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7af270eb5c214fcb9395f8906459f08a" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 33403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:15:46.030234 13462 mvcc.cc:205] Tried to move back new op lower bound from 7315664674893037568 to 7315664674726850560. Current Snapshot: MvccSnapshot[applied={T|T < 7315664674893037568}]
I20260806 21:15:46.030596 13464 mvcc.cc:205] Tried to move back new op lower bound from 7315664674893037568 to 7315664674726850560. Current Snapshot: MvccSnapshot[applied={T|T < 7315664674893037568}]
I20260806 21:15:46.040305 12771 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52826:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:15:46.040529 12771 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52826:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:15:46.043445 12771 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0f13ffde91aa4442853c38269107a5bc: Sending DeleteTablet for 3 replicas of tablet 53a36d144e854b09a60e87ece224f97e
I20260806 21:15:46.044814 13127 tablet_service.cc:1514] Processing DeleteTablet for tablet 53a36d144e854b09a60e87ece224f97e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:15:46 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:55090
I20260806 21:15:46.044818 13323 tablet_service.cc:1514] Processing DeleteTablet for tablet 53a36d144e854b09a60e87ece224f97e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:15:46 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:48492
I20260806 21:15:46.044837 12942 tablet_service.cc:1514] Processing DeleteTablet for tablet 53a36d144e854b09a60e87ece224f97e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:15:46 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:48074
I20260806 21:15:46.045763 13479 tablet_replica.cc:330] T 53a36d144e854b09a60e87ece224f97e P 4b4b603b7e7141b8aef75e5b6db45e33: stopping tablet replica
I20260806 21:15:46.046186 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 12859
I20260806 21:15:46.058048 13479 raft_consensus.cc:2229] T 53a36d144e854b09a60e87ece224f97e P 4b4b603b7e7141b8aef75e5b6db45e33 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:15:46.058485 13479 raft_consensus.cc:2258] T 53a36d144e854b09a60e87ece224f97e P 4b4b603b7e7141b8aef75e5b6db45e33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:15:46.058990 13476 tablet_replica.cc:330] T 53a36d144e854b09a60e87ece224f97e P 7af270eb5c214fcb9395f8906459f08a: stopping tablet replica
I20260806 21:15:46.059336 13476 raft_consensus.cc:2229] T 53a36d144e854b09a60e87ece224f97e P 7af270eb5c214fcb9395f8906459f08a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:15:46.059639 13476 raft_consensus.cc:2258] T 53a36d144e854b09a60e87ece224f97e P 7af270eb5c214fcb9395f8906459f08a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:15:46.060729 13479 ts_tablet_manager.cc:1905] T 53a36d144e854b09a60e87ece224f97e P 4b4b603b7e7141b8aef75e5b6db45e33: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:15:46.061005 13476 ts_tablet_manager.cc:1905] T 53a36d144e854b09a60e87ece224f97e P 7af270eb5c214fcb9395f8906459f08a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:15:46.063495 13476 ts_tablet_manager.cc:1918] T 53a36d144e854b09a60e87ece224f97e P 7af270eb5c214fcb9395f8906459f08a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:15:46.063613 13476 log.cc:1196] T 53a36d144e854b09a60e87ece224f97e P 7af270eb5c214fcb9395f8906459f08a: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-1/wal/wals/53a36d144e854b09a60e87ece224f97e
I20260806 21:15:46.063719 13479 ts_tablet_manager.cc:1918] T 53a36d144e854b09a60e87ece224f97e P 4b4b603b7e7141b8aef75e5b6db45e33: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:15:46.063836 13479 log.cc:1196] T 53a36d144e854b09a60e87ece224f97e P 4b4b603b7e7141b8aef75e5b6db45e33: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786050686797304-17096-0/minicluster-data/ts-2/wal/wals/53a36d144e854b09a60e87ece224f97e
I20260806 21:15:46.063946 13476 ts_tablet_manager.cc:1939] T 53a36d144e854b09a60e87ece224f97e P 7af270eb5c214fcb9395f8906459f08a: Deleting consensus metadata
I20260806 21:15:46.064206 13479 ts_tablet_manager.cc:1939] T 53a36d144e854b09a60e87ece224f97e P 4b4b603b7e7141b8aef75e5b6db45e33: Deleting consensus metadata
I20260806 21:15:46.065500 12752 catalog_manager.cc:4841] TS 7af270eb5c214fcb9395f8906459f08a (127.16.178.2:33403): tablet 53a36d144e854b09a60e87ece224f97e (table test-table [id=291792fb3ac649bd8e9ce9951ffe102e]) successfully deleted
I20260806 21:15:46.065639 12753 catalog_manager.cc:4841] TS 4b4b603b7e7141b8aef75e5b6db45e33 (127.16.178.3:35273): tablet 53a36d144e854b09a60e87ece224f97e (table test-table [id=291792fb3ac649bd8e9ce9951ffe102e]) successfully deleted
W20260806 21:15:46.066284 12756 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.178.1:37951 (error 108)
I20260806 21:15:46.067191 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 13039
I20260806 21:15:46.068375 13433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:46.067108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:37951 (local address 127.0.0.1:48094)
0806 21:15:46.068217 (+ 1109us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:46.068288 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.1:37951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1013}
W20260806 21:15:46.070116 12756 catalog_manager.cc:4584] TS c25d6a6dc6aa4c7ba369cb2ba1621821 (127.16.178.1:37951): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 53a36d144e854b09a60e87ece224f97e: Network error: Client connection negotiation failed: client connection to 127.16.178.1:37951: connect: Connection refused (error 111)
I20260806 21:15:46.082352 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 13232
I20260806 21:15:46.102276 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 12732
2026-08-06T21:15:46Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2538 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-06T21:15:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:15:46Z Disabled control of system clock
I20260806 21:15:46.148236 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:38065
--webserver_interface=127.16.178.62
--webserver_port=0
--builtin_ntp_servers=127.16.178.20:46441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:38065
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 21:15:46.265460 13487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:15:46.265758 13487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:15:46.265810 13487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:15:46.269873 13487 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:15:46.269950 13487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:15:46.269973 13487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:15:46.269994 13487 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:15:46.270015 13487 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:15:46.274950 13487 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:46441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:38065
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:38065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.178.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:15:46.276183 13487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:15:46.277640 13487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:15:46.286809 13497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:46.286911 13495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:46.286844 13500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:46.287078 13487 server_base.cc:1034] running on GCE node
I20260806 21:15:46.287747 13487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:15:46.289228 13487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:15:46.290442 13487 hybrid_clock.cc:648] HybridClock initialized: now 1786050946290422 us; error 41 us; skew 500 ppm
I20260806 21:15:46.292853 13487 webserver.cc:460] Webserver started at http://127.16.178.62:42793/ using document root <none> and password file <none>
I20260806 21:15:46.293581 13487 fs_manager.cc:359] Metadata directory not provided
I20260806 21:15:46.293674 13487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:15:46.293926 13487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:15:46.296437 13487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "f2470ddce0594bf4a4d5f7fa70e19863"
format_stamp: "Formatted at 2026-08-06 21:15:46 on dist-test-slave-lgr5"
server_key: "8ffea86f31d95f6e85e7870160ab430a"
server_key_iv: "57e4175667ceaa14a37cdaac3ea9c041"
server_key_version: "encryptionkey@0"
I20260806 21:15:46.297109 13487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "f2470ddce0594bf4a4d5f7fa70e19863"
format_stamp: "Formatted at 2026-08-06 21:15:46 on dist-test-slave-lgr5"
server_key: "8ffea86f31d95f6e85e7870160ab430a"
server_key_iv: "57e4175667ceaa14a37cdaac3ea9c041"
server_key_version: "encryptionkey@0"
I20260806 21:15:46.301934 13487 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 21:15:46.305838 13506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:46.307742 13487 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:15:46.307978 13487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "f2470ddce0594bf4a4d5f7fa70e19863"
format_stamp: "Formatted at 2026-08-06 21:15:46 on dist-test-slave-lgr5"
server_key: "8ffea86f31d95f6e85e7870160ab430a"
server_key_iv: "57e4175667ceaa14a37cdaac3ea9c041"
server_key_version: "encryptionkey@0"
I20260806 21:15:46.308197 13487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:15:46.339699 13487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:15:46.340521 13487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:15:46.340710 13487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:15:46.348270 13487 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.62:38065
I20260806 21:15:46.348289 13587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.62:38065 every 8 connection(s)
I20260806 21:15:46.349398 13487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:15:46.353127 13588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:15:46.355829 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 13487
I20260806 21:15:46.355978 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:15:46.356338 17096 external_mini_cluster.cc:1428] Setting key a5d482451bf37544afcdad2b4a816920
I20260806 21:15:46.359747 13588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2470ddce0594bf4a4d5f7fa70e19863: Bootstrap starting.
I20260806 21:15:46.362135 13588 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2470ddce0594bf4a4d5f7fa70e19863: Neither blocks nor log segments found. Creating new log.
I20260806 21:15:46.362913 13588 log.cc:824] T 00000000000000000000000000000000 P f2470ddce0594bf4a4d5f7fa70e19863: Log is configured to *not* fsync() on all Append() calls
I20260806 21:15:46.365265 13588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2470ddce0594bf4a4d5f7fa70e19863: No bootstrap required, opened a new log
I20260806 21:15:46.365979 13591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:46.357903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51498 (local address 127.16.178.62:38065)
0806 21:15:46.358627 (+ 724us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:46.358643 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:46.358681 (+ 38us) server_negotiation.cc:407] Connection header received
0806 21:15:46.359311 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:46.359329 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:46.359756 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:46.360127 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:15:46.360592 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:46.361586 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:46.362508 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:46.363000 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:46.363154 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:46.364129 (+ 975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:15:46.364135 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:15:46.364151 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:15:46.364571 (+ 420us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:15:46.364632 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:15:46.364655 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:15:46.364742 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:46.364902 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 21:15:46.365209 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":142,"threads_started":1}
I20260806 21:15:46.369055 13588 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2470ddce0594bf4a4d5f7fa70e19863 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2470ddce0594bf4a4d5f7fa70e19863" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 38065 } }
I20260806 21:15:46.369297 13588 raft_consensus.cc:374] T 00000000000000000000000000000000 P f2470ddce0594bf4a4d5f7fa70e19863 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:15:46.369343 13588 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2470ddce0594bf4a4d5f7fa70e19863 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2470ddce0594bf4a4d5f7fa70e19863, State: Initialized, Role: FOLLOWER
I20260806 21:15:46.369791 13588 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2470ddce0594bf4a4d5f7fa70e19863 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2470ddce0594bf4a4d5f7fa70e19863" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 38065 } }
I20260806 21:15:46.369930 13588 raft_consensus.cc:388] T 00000000000000000000000000000000 P f2470ddce0594bf4a4d5f7fa70e19863 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:15:46.369995 13588 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2470ddce0594bf4a4d5f7fa70e19863 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:15:46.370097 13588 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2470ddce0594bf4a4d5f7fa70e19863 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:15:46.371371 13588 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2470ddce0594bf4a4d5f7fa70e19863 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2470ddce0594bf4a4d5f7fa70e19863" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 38065 } }
I20260806 21:15:46.371738 13588 leader_election.cc:302] T 00000000000000000000000000000000 P f2470ddce0594bf4a4d5f7fa70e19863 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f2470ddce0594bf4a4d5f7fa70e19863; no voters:
I20260806 21:15:46.372069 13588 leader_election.cc:288] T 00000000000000000000000000000000 P f2470ddce0594bf4a4d5f7fa70e19863 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:15:46.372202 13593 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f2470ddce0594bf4a4d5f7fa70e19863 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:15:46.372417 13593 raft_consensus.cc:686] T 00000000000000000000000000000000 P f2470ddce0594bf4a4d5f7fa70e19863 [term 1 LEADER]: Becoming Leader. State: Replica: f2470ddce0594bf4a4d5f7fa70e19863, State: Running, Role: LEADER
I20260806 21:15:46.372753 13593 consensus_queue.cc:237] T 00000000000000000000000000000000 P f2470ddce0594bf4a4d5f7fa70e19863 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2470ddce0594bf4a4d5f7fa70e19863" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 38065 } }
I20260806 21:15:46.373412 13588 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2470ddce0594bf4a4d5f7fa70e19863 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:15:46.374472 13596 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2470ddce0594bf4a4d5f7fa70e19863 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2470ddce0594bf4a4d5f7fa70e19863. Latest consensus state: current_term: 1 leader_uuid: "f2470ddce0594bf4a4d5f7fa70e19863" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2470ddce0594bf4a4d5f7fa70e19863" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 38065 } } }
I20260806 21:15:46.374629 13596 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2470ddce0594bf4a4d5f7fa70e19863 [sys.catalog]: This master's current role is: LEADER
I20260806 21:15:46.374907 13595 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2470ddce0594bf4a4d5f7fa70e19863 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f2470ddce0594bf4a4d5f7fa70e19863" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2470ddce0594bf4a4d5f7fa70e19863" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 38065 } } }
I20260806 21:15:46.375015 13595 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2470ddce0594bf4a4d5f7fa70e19863 [sys.catalog]: This master's current role is: LEADER
I20260806 21:15:46.375172 13606 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:15:46.378451 13606 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:15:46.385226 13606 catalog_manager.cc:1347] Generated new cluster ID: 979c73511c4141cb9e91e1bdaa34339a
I20260806 21:15:46.385313 13606 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:15:46.398540 13606 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:15:46.399962 13606 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:15:46.411404 13606 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f2470ddce0594bf4a4d5f7fa70e19863: Generated new TSK 0
I20260806 21:15:46.412209 13606 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:15:46.424325 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.1:0
--local_ip_for_outbound_sockets=127.16.178.1
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:38065
--builtin_ntp_servers=127.16.178.20:46441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 21:15:46.543923 13619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:15:46.544221 13619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:15:46.544281 13619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:15:46.548262 13619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:15:46.548331 13619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:15:46.548400 13619 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.1
I20260806 21:15:46.553308 13619 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:46441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:38065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13619
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.1
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:15:46.554438 13619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:15:46.555895 13619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:15:46.565580 13626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:46.565629 13628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:46.565598 13632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:46.565646 13619 server_base.cc:1034] running on GCE node
I20260806 21:15:46.566252 13619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:15:46.566942 13619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:15:46.568128 13619 hybrid_clock.cc:648] HybridClock initialized: now 1786050946568111 us; error 35 us; skew 500 ppm
I20260806 21:15:46.570065 13619 webserver.cc:460] Webserver started at http://127.16.178.1:42157/ using document root <none> and password file <none>
I20260806 21:15:46.570658 13619 fs_manager.cc:359] Metadata directory not provided
I20260806 21:15:46.570727 13619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:15:46.570940 13619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:15:46.573143 13619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-0/data/instance:
uuid: "e879b1bdf4614d5aa43258462e7a0c40"
format_stamp: "Formatted at 2026-08-06 21:15:46 on dist-test-slave-lgr5"
server_key: "1698da47ad6b34b58c0f8816447a7a7f"
server_key_iv: "3ff9e06e16620e2000b1b815329d3c8d"
server_key_version: "encryptionkey@0"
I20260806 21:15:46.573743 13619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance:
uuid: "e879b1bdf4614d5aa43258462e7a0c40"
format_stamp: "Formatted at 2026-08-06 21:15:46 on dist-test-slave-lgr5"
server_key: "1698da47ad6b34b58c0f8816447a7a7f"
server_key_iv: "3ff9e06e16620e2000b1b815329d3c8d"
server_key_version: "encryptionkey@0"
I20260806 21:15:46.578050 13619 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:15:46.580845 13639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:46.582204 13619 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:15:46.582332 13619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-0/wal
uuid: "e879b1bdf4614d5aa43258462e7a0c40"
format_stamp: "Formatted at 2026-08-06 21:15:46 on dist-test-slave-lgr5"
server_key: "1698da47ad6b34b58c0f8816447a7a7f"
server_key_iv: "3ff9e06e16620e2000b1b815329d3c8d"
server_key_version: "encryptionkey@0"
I20260806 21:15:46.582450 13619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:15:46.594396 13619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:15:46.595124 13619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:15:46.595302 13619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:15:46.595909 13619 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:15:46.597028 13619 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:15:46.597080 13619 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:46.597136 13619 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:15:46.597160 13619 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:46.606386 13619 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.1:46735
I20260806 21:15:46.606415 13802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.1:46735 every 8 connection(s)
I20260806 21:15:46.607434 13619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
I20260806 21:15:46.612260 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 13619
I20260806 21:15:46.612411 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance
I20260806 21:15:46.612746 17096 external_mini_cluster.cc:1428] Setting key 3cb2f06d87411e9fa625a23c6e505055
I20260806 21:15:46.614847 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.2:0
--local_ip_for_outbound_sockets=127.16.178.2
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:38065
--builtin_ntp_servers=127.16.178.20:46441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 21:15:46.617586 13591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:46.609451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:46287 (local address 127.16.178.62:38065)
0806 21:15:46.609630 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:46.609635 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:46.610766 (+ 1131us) server_negotiation.cc:407] Connection header received
0806 21:15:46.612267 (+ 1501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:46.612270 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:46.612324 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:46.612415 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:15:46.613595 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:46.614298 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:46.615544 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:46.615842 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:46.615956 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:46.617123 (+ 1167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:15:46.617127 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:15:46.617130 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:15:46.617145 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:15:46.617168 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:15:46.617178 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:15:46.617243 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:46.617401 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 21:15:46.617462 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 21:15:46.618649 13811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:46.609904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:38065 (local address 127.16.178.1:46287)
0806 21:15:46.610610 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:46.610655 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:46.611980 (+ 1325us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:46.612719 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:46.612728 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:46.612834 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:15:46.613466 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:46.613484 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:46.614435 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:46.614438 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:46.615404 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:46.615423 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:46.615984 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:46.615987 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:46.616171 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:46.616787 (+ 616us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 21:15:46.616789 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:15:46.616851 (+ 62us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 21:15:46.616942 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:15:46.617254 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:15:46.617261 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:15:46.617267 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 21:15:46.617391 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 21:15:46.617723 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":126,"threads_started":1}
I20260806 21:15:46.619828 13803 heartbeater.cc:347] Connected to a master server at 127.16.178.62:38065
I20260806 21:15:46.620070 13803 heartbeater.cc:464] Registering TS with master...
I20260806 21:15:46.620635 13803 heartbeater.cc:511] Master 127.16.178.62:38065 requested a full tablet report, sending...
I20260806 21:15:46.622157 13536 ts_manager.cc:194] Registered new tserver with Master: e879b1bdf4614d5aa43258462e7a0c40 (127.16.178.1:46735)
I20260806 21:15:46.623301 13536 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.178.1:46287
W20260806 21:15:46.732620 13814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:15:46.732909 13814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:15:46.732970 13814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:15:46.737030 13814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:15:46.737093 13814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:15:46.737176 13814 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.2
I20260806 21:15:46.743093 13814 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:46441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:38065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13814
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.2
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:15:46.744335 13814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:15:46.745791 13814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 21:15:46.756937 13814 server_base.cc:1034] running on GCE node
W20260806 21:15:46.757169 13823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:46.757347 13821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:46.760977 13820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:46.761648 13814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:15:46.762534 13814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:15:46.763723 13814 hybrid_clock.cc:648] HybridClock initialized: now 1786050946763700 us; error 70 us; skew 500 ppm
I20260806 21:15:46.766134 13814 webserver.cc:460] Webserver started at http://127.16.178.2:34479/ using document root <none> and password file <none>
I20260806 21:15:46.766803 13814 fs_manager.cc:359] Metadata directory not provided
I20260806 21:15:46.766872 13814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:15:46.767081 13814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:15:46.769377 13814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-1/data/instance:
uuid: "f290d0eb984f4c6d8cef0e5c64c1380d"
format_stamp: "Formatted at 2026-08-06 21:15:46 on dist-test-slave-lgr5"
server_key: "8c3a6b3c07a9f36663247fb275824d1b"
server_key_iv: "a48d82286f08db380679d2eb6a34540e"
server_key_version: "encryptionkey@0"
I20260806 21:15:46.769932 13814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance:
uuid: "f290d0eb984f4c6d8cef0e5c64c1380d"
format_stamp: "Formatted at 2026-08-06 21:15:46 on dist-test-slave-lgr5"
server_key: "8c3a6b3c07a9f36663247fb275824d1b"
server_key_iv: "a48d82286f08db380679d2eb6a34540e"
server_key_version: "encryptionkey@0"
I20260806 21:15:46.774222 13814 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260806 21:15:46.777511 13832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:46.779101 13814 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:15:46.779247 13814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-1/wal
uuid: "f290d0eb984f4c6d8cef0e5c64c1380d"
format_stamp: "Formatted at 2026-08-06 21:15:46 on dist-test-slave-lgr5"
server_key: "8c3a6b3c07a9f36663247fb275824d1b"
server_key_iv: "a48d82286f08db380679d2eb6a34540e"
server_key_version: "encryptionkey@0"
I20260806 21:15:46.779385 13814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:15:46.806099 13814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:15:46.806854 13814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:15:46.807049 13814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:15:46.807680 13814 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:15:46.808789 13814 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:15:46.808844 13814 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:46.808902 13814 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:15:46.808928 13814 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:46.819067 13814 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.2:32851
I20260806 21:15:46.819131 13991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.2:32851 every 8 connection(s)
I20260806 21:15:46.820127 13814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
I20260806 21:15:46.821270 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 13814
I20260806 21:15:46.821395 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance
I20260806 21:15:46.821689 17096 external_mini_cluster.cc:1428] Setting key a61041162d83d94c490e55985fa86731
I20260806 21:15:46.823908 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.3:0
--local_ip_for_outbound_sockets=127.16.178.3
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:38065
--builtin_ntp_servers=127.16.178.20:46441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 21:15:46.831053 13591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:46.822366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:35425 (local address 127.16.178.62:38065)
0806 21:15:46.824173 (+ 1807us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:46.824178 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:46.824200 (+ 22us) server_negotiation.cc:407] Connection header received
0806 21:15:46.825960 (+ 1760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:46.825964 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:46.826033 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:46.826077 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:15:46.827272 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:46.827820 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:46.829065 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:46.829298 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:46.829390 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:46.830531 (+ 1141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:15:46.830536 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:15:46.830538 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:15:46.830554 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:15:46.830576 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:15:46.830584 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:15:46.830644 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:46.830798 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 21:15:46.830873 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1676}
I20260806 21:15:46.832060 13995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:46.822953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:38065 (local address 127.16.178.2:35425)
0806 21:15:46.823910 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:46.823978 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:46.825670 (+ 1692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:46.826424 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:46.826430 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:46.826553 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:15:46.827123 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:46.827137 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:46.827945 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:46.827948 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:46.828942 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:46.828954 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:46.829450 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:46.829455 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:46.829625 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:46.830241 (+ 616us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 21:15:46.830243 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:15:46.830287 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 21:15:46.830362 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:15:46.830655 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:15:46.830665 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:15:46.830672 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 21:15:46.830794 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 21:15:46.831153 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":198,"threads_started":1}
I20260806 21:15:46.833400 13993 heartbeater.cc:347] Connected to a master server at 127.16.178.62:38065
I20260806 21:15:46.833679 13993 heartbeater.cc:464] Registering TS with master...
I20260806 21:15:46.834319 13993 heartbeater.cc:511] Master 127.16.178.62:38065 requested a full tablet report, sending...
I20260806 21:15:46.835538 13536 ts_manager.cc:194] Registered new tserver with Master: f290d0eb984f4c6d8cef0e5c64c1380d (127.16.178.2:32851)
I20260806 21:15:46.836213 13536 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.178.2:35425
W20260806 21:15:46.951958 13996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:15:46.952304 13996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:15:46.952349 13996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:15:46.956513 13996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:15:46.956575 13996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:15:46.956631 13996 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.3
I20260806 21:15:46.961565 13996 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:46441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:38065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13996
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.3
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:15:46.962754 13996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:15:46.964252 13996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:15:46.974876 14006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:46.974876 14004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:46.975131 13996 server_base.cc:1034] running on GCE node
W20260806 21:15:46.974889 14003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:46.975715 13996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:15:46.976537 13996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:15:46.977705 13996 hybrid_clock.cc:648] HybridClock initialized: now 1786050946977682 us; error 40 us; skew 500 ppm
I20260806 21:15:46.979664 13996 webserver.cc:460] Webserver started at http://127.16.178.3:37451/ using document root <none> and password file <none>
I20260806 21:15:46.980319 13996 fs_manager.cc:359] Metadata directory not provided
I20260806 21:15:46.980392 13996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:15:46.980613 13996 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:15:46.982913 13996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-2/data/instance:
uuid: "7ec42ff6d2874d2c9cfd308ea7139a35"
format_stamp: "Formatted at 2026-08-06 21:15:46 on dist-test-slave-lgr5"
server_key: "a0d6e7955e4a4da73e43ef528fa3a2f0"
server_key_iv: "afc44fbe1afeb412dfc2de9384593359"
server_key_version: "encryptionkey@0"
I20260806 21:15:46.983556 13996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance:
uuid: "7ec42ff6d2874d2c9cfd308ea7139a35"
format_stamp: "Formatted at 2026-08-06 21:15:46 on dist-test-slave-lgr5"
server_key: "a0d6e7955e4a4da73e43ef528fa3a2f0"
server_key_iv: "afc44fbe1afeb412dfc2de9384593359"
server_key_version: "encryptionkey@0"
I20260806 21:15:46.987923 13996 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260806 21:15:46.990586 14015 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:46.991827 13996 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 21:15:46.991942 13996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-2/wal
uuid: "7ec42ff6d2874d2c9cfd308ea7139a35"
format_stamp: "Formatted at 2026-08-06 21:15:46 on dist-test-slave-lgr5"
server_key: "a0d6e7955e4a4da73e43ef528fa3a2f0"
server_key_iv: "afc44fbe1afeb412dfc2de9384593359"
server_key_version: "encryptionkey@0"
I20260806 21:15:46.992168 13996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:15:47.004128 13996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:15:47.004865 13996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:15:47.005084 13996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:15:47.005709 13996 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:15:47.006731 13996 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:15:47.006778 13996 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:47.006824 13996 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:15:47.006852 13996 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:47.016553 13996 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.3:34739
I20260806 21:15:47.016624 14191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.3:34739 every 8 connection(s)
I20260806 21:15:47.017644 13996 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
I20260806 21:15:47.022850 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 13996
I20260806 21:15:47.023007 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance
I20260806 21:15:47.023293 17096 external_mini_cluster.cc:1428] Setting key 8afccdbf7460678d1469c578a58988da
I20260806 21:15:47.028743 13591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:47.019864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:47475 (local address 127.16.178.62:38065)
0806 21:15:47.020069 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:47.020075 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:47.021310 (+ 1235us) server_negotiation.cc:407] Connection header received
0806 21:15:47.022924 (+ 1614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:47.022927 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:47.023003 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:47.023120 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:15:47.024498 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:47.025256 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:47.026666 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:47.026941 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:47.027073 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:47.028278 (+ 1205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:15:47.028281 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:15:47.028284 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:15:47.028297 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:15:47.028319 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:15:47.028327 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:15:47.028388 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:47.028566 (+ 178us) server_negotiation.cc:299] Negotiation successful
0806 21:15:47.028623 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 21:15:47.029632 14197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:47.020348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:38065 (local address 127.16.178.3:47475)
0806 21:15:47.021139 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:47.021186 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:47.022636 (+ 1450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:47.023360 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:47.023371 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:47.023487 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:15:47.024306 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:47.024328 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:47.025455 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:47.025458 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:47.026532 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:47.026544 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:47.027133 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:47.027137 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:47.027305 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:47.027959 (+ 654us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 21:15:47.027960 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:15:47.028006 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 21:15:47.028108 (+ 102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:15:47.028402 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:15:47.028429 (+ 27us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:15:47.028434 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 21:15:47.028544 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 21:15:47.028790 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":136,"threads_started":1}
I20260806 21:15:47.030762 14195 heartbeater.cc:347] Connected to a master server at 127.16.178.62:38065
I20260806 21:15:47.031033 14195 heartbeater.cc:464] Registering TS with master...
I20260806 21:15:47.031637 14195 heartbeater.cc:511] Master 127.16.178.62:38065 requested a full tablet report, sending...
I20260806 21:15:47.032881 13536 ts_manager.cc:194] Registered new tserver with Master: 7ec42ff6d2874d2c9cfd308ea7139a35 (127.16.178.3:34739)
I20260806 21:15:47.033500 13536 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.178.3:47475
I20260806 21:15:47.037945 17096 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 21:15:47.044004 13591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:47.040052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51502 (local address 127.16.178.62:38065)
0806 21:15:47.040211 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:47.040216 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:47.040412 (+ 196us) server_negotiation.cc:407] Connection header received
0806 21:15:47.040539 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:47.040543 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:47.040595 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:47.040661 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:15:47.040953 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:47.041503 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:47.042335 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:47.042562 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:47.042651 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:47.043492 (+ 841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:15:47.043496 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:15:47.043499 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:15:47.043513 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:15:47.043534 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:15:47.043543 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:15:47.043651 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:47.043801 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 21:15:47.043844 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 21:15:47.047362 13536 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51502:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:15:47.050990 13536 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:15:47.066006 14207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:47.060251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.3:34739 (local address 127.0.0.1:56578)
0806 21:15:47.060720 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:47.060778 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:47.060917 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:47.061728 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:47.061734 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:47.061759 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:15:47.061913 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:47.061921 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:47.063024 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:47.063027 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:47.064467 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:47.064479 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:47.065545 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:47.065549 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:47.065655 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:47.065708 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 21:15:47.065784 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 21:15:47.065877 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":149,"threads_started":1}
I20260806 21:15:47.066931 14212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:47.061627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45188 (local address 127.16.178.1:46735)
0806 21:15:47.062854 (+ 1227us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:47.062862 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:47.062888 (+ 26us) server_negotiation.cc:407] Connection header received
0806 21:15:47.062948 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:47.062952 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:47.063121 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:47.063265 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:15:47.063506 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:47.064628 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:47.065562 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:47.066328 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:47.066430 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:47.066610 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:47.066660 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 21:15:47.066752 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1123,"thread_start_us":142,"threads_started":1}
I20260806 21:15:47.066949 14208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:47.060394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56578 (local address 127.16.178.3:34739)
0806 21:15:47.061006 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:47.061020 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:47.061047 (+ 27us) server_negotiation.cc:407] Connection header received
0806 21:15:47.061133 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:47.061139 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:47.061371 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:47.061532 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:15:47.062062 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:47.062909 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:47.064581 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:47.065395 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:47.066227 (+ 832us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:47.066444 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:47.066496 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 21:15:47.066781 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":125,"threads_started":1}
I20260806 21:15:47.067265 14211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:47.061453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:46735 (local address 127.0.0.1:45188)
0806 21:15:47.061918 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:47.061931 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:47.062006 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:47.063269 (+ 1263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:47.063272 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:47.063294 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:15:47.063384 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:47.063390 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:47.064721 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:47.064724 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:47.065439 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:47.065448 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:47.066435 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:47.066439 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:47.066538 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:47.066561 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 21:15:47.066624 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 21:15:47.066685 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":90,"threads_started":1}
I20260806 21:15:47.067770 14213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:47.061884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49006 (local address 127.16.178.2:32851)
0806 21:15:47.062197 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:47.062206 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:47.062226 (+ 20us) server_negotiation.cc:407] Connection header received
0806 21:15:47.062284 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:47.062288 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:47.062423 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:47.062547 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:15:47.062791 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:47.064396 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:47.065716 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:47.066719 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:47.066818 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:47.066986 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:47.067525 (+ 539us) server_negotiation.cc:299] Negotiation successful
0806 21:15:47.067614 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":92,"threads_started":1}
I20260806 21:15:47.067427 14210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:47.061107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.2:32851 (local address 127.0.0.1:49006)
0806 21:15:47.061340 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:47.061352 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:47.061436 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:47.062547 (+ 1111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:47.062550 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:47.062568 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:15:47.062658 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:47.062664 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:47.064508 (+ 1844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:47.064512 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:47.065597 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:47.065606 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:47.067055 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:47.067059 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:47.067142 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:47.067165 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 21:15:47.067228 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 21:15:47.067292 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":112,"threads_started":1}
I20260806 21:15:47.070101 13716 tablet_service.cc:1467] Processing CreateTablet for tablet 714cecfd274f4a42bbbb5f9a405751d0 (DEFAULT_TABLE table=test-table [id=9caab356bf634e6bb4e45fcda3633a78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:15:47.070098 14091 tablet_service.cc:1467] Processing CreateTablet for tablet 714cecfd274f4a42bbbb5f9a405751d0 (DEFAULT_TABLE table=test-table [id=9caab356bf634e6bb4e45fcda3633a78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:15:47.071177 14091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 714cecfd274f4a42bbbb5f9a405751d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:15:47.071178 13716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 714cecfd274f4a42bbbb5f9a405751d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:15:47.070963 13901 tablet_service.cc:1467] Processing CreateTablet for tablet 714cecfd274f4a42bbbb5f9a405751d0 (DEFAULT_TABLE table=test-table [id=9caab356bf634e6bb4e45fcda3633a78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:15:47.072009 13901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 714cecfd274f4a42bbbb5f9a405751d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:15:47.078184 14214 tablet_bootstrap.cc:492] T 714cecfd274f4a42bbbb5f9a405751d0 P e879b1bdf4614d5aa43258462e7a0c40: Bootstrap starting.
I20260806 21:15:47.079511 14215 tablet_bootstrap.cc:492] T 714cecfd274f4a42bbbb5f9a405751d0 P 7ec42ff6d2874d2c9cfd308ea7139a35: Bootstrap starting.
I20260806 21:15:47.080452 14216 tablet_bootstrap.cc:492] T 714cecfd274f4a42bbbb5f9a405751d0 P f290d0eb984f4c6d8cef0e5c64c1380d: Bootstrap starting.
I20260806 21:15:47.080852 14214 tablet_bootstrap.cc:654] T 714cecfd274f4a42bbbb5f9a405751d0 P e879b1bdf4614d5aa43258462e7a0c40: Neither blocks nor log segments found. Creating new log.
I20260806 21:15:47.081619 14215 tablet_bootstrap.cc:654] T 714cecfd274f4a42bbbb5f9a405751d0 P 7ec42ff6d2874d2c9cfd308ea7139a35: Neither blocks nor log segments found. Creating new log.
I20260806 21:15:47.081626 14214 log.cc:824] T 714cecfd274f4a42bbbb5f9a405751d0 P e879b1bdf4614d5aa43258462e7a0c40: Log is configured to *not* fsync() on all Append() calls
I20260806 21:15:47.082475 14215 log.cc:824] T 714cecfd274f4a42bbbb5f9a405751d0 P 7ec42ff6d2874d2c9cfd308ea7139a35: Log is configured to *not* fsync() on all Append() calls
I20260806 21:15:47.082576 14216 tablet_bootstrap.cc:654] T 714cecfd274f4a42bbbb5f9a405751d0 P f290d0eb984f4c6d8cef0e5c64c1380d: Neither blocks nor log segments found. Creating new log.
I20260806 21:15:47.083420 14216 log.cc:824] T 714cecfd274f4a42bbbb5f9a405751d0 P f290d0eb984f4c6d8cef0e5c64c1380d: Log is configured to *not* fsync() on all Append() calls
I20260806 21:15:47.084481 14214 tablet_bootstrap.cc:492] T 714cecfd274f4a42bbbb5f9a405751d0 P e879b1bdf4614d5aa43258462e7a0c40: No bootstrap required, opened a new log
I20260806 21:15:47.084646 14215 tablet_bootstrap.cc:492] T 714cecfd274f4a42bbbb5f9a405751d0 P 7ec42ff6d2874d2c9cfd308ea7139a35: No bootstrap required, opened a new log
I20260806 21:15:47.084713 14214 ts_tablet_manager.cc:1397] T 714cecfd274f4a42bbbb5f9a405751d0 P e879b1bdf4614d5aa43258462e7a0c40: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.001s
I20260806 21:15:47.084792 14215 ts_tablet_manager.cc:1397] T 714cecfd274f4a42bbbb5f9a405751d0 P 7ec42ff6d2874d2c9cfd308ea7139a35: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260806 21:15:47.085243 14216 tablet_bootstrap.cc:492] T 714cecfd274f4a42bbbb5f9a405751d0 P f290d0eb984f4c6d8cef0e5c64c1380d: No bootstrap required, opened a new log
I20260806 21:15:47.085433 14216 ts_tablet_manager.cc:1397] T 714cecfd274f4a42bbbb5f9a405751d0 P f290d0eb984f4c6d8cef0e5c64c1380d: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260806 21:15:47.088697 14216 raft_consensus.cc:348] T 714cecfd274f4a42bbbb5f9a405751d0 P f290d0eb984f4c6d8cef0e5c64c1380d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec42ff6d2874d2c9cfd308ea7139a35" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 34739 } } peers { permanent_uuid: "f290d0eb984f4c6d8cef0e5c64c1380d" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 32851 } } peers { permanent_uuid: "e879b1bdf4614d5aa43258462e7a0c40" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 46735 } }
I20260806 21:15:47.088941 14216 raft_consensus.cc:374] T 714cecfd274f4a42bbbb5f9a405751d0 P f290d0eb984f4c6d8cef0e5c64c1380d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:15:47.089003 14216 raft_consensus.cc:729] T 714cecfd274f4a42bbbb5f9a405751d0 P f290d0eb984f4c6d8cef0e5c64c1380d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f290d0eb984f4c6d8cef0e5c64c1380d, State: Initialized, Role: FOLLOWER
I20260806 21:15:47.089195 14214 raft_consensus.cc:348] T 714cecfd274f4a42bbbb5f9a405751d0 P e879b1bdf4614d5aa43258462e7a0c40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec42ff6d2874d2c9cfd308ea7139a35" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 34739 } } peers { permanent_uuid: "f290d0eb984f4c6d8cef0e5c64c1380d" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 32851 } } peers { permanent_uuid: "e879b1bdf4614d5aa43258462e7a0c40" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 46735 } }
I20260806 21:15:47.089457 14214 raft_consensus.cc:374] T 714cecfd274f4a42bbbb5f9a405751d0 P e879b1bdf4614d5aa43258462e7a0c40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:15:47.089527 14214 raft_consensus.cc:729] T 714cecfd274f4a42bbbb5f9a405751d0 P e879b1bdf4614d5aa43258462e7a0c40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e879b1bdf4614d5aa43258462e7a0c40, State: Initialized, Role: FOLLOWER
I20260806 21:15:47.089461 14216 consensus_queue.cc:260] T 714cecfd274f4a42bbbb5f9a405751d0 P f290d0eb984f4c6d8cef0e5c64c1380d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec42ff6d2874d2c9cfd308ea7139a35" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 34739 } } peers { permanent_uuid: "f290d0eb984f4c6d8cef0e5c64c1380d" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 32851 } } peers { permanent_uuid: "e879b1bdf4614d5aa43258462e7a0c40" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 46735 } }
I20260806 21:15:47.089784 14215 raft_consensus.cc:348] T 714cecfd274f4a42bbbb5f9a405751d0 P 7ec42ff6d2874d2c9cfd308ea7139a35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec42ff6d2874d2c9cfd308ea7139a35" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 34739 } } peers { permanent_uuid: "f290d0eb984f4c6d8cef0e5c64c1380d" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 32851 } } peers { permanent_uuid: "e879b1bdf4614d5aa43258462e7a0c40" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 46735 } }
I20260806 21:15:47.090077 14215 raft_consensus.cc:374] T 714cecfd274f4a42bbbb5f9a405751d0 P 7ec42ff6d2874d2c9cfd308ea7139a35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:15:47.089996 14214 consensus_queue.cc:260] T 714cecfd274f4a42bbbb5f9a405751d0 P e879b1bdf4614d5aa43258462e7a0c40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec42ff6d2874d2c9cfd308ea7139a35" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 34739 } } peers { permanent_uuid: "f290d0eb984f4c6d8cef0e5c64c1380d" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 32851 } } peers { permanent_uuid: "e879b1bdf4614d5aa43258462e7a0c40" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 46735 } }
I20260806 21:15:47.090185 14215 raft_consensus.cc:729] T 714cecfd274f4a42bbbb5f9a405751d0 P 7ec42ff6d2874d2c9cfd308ea7139a35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ec42ff6d2874d2c9cfd308ea7139a35, State: Initialized, Role: FOLLOWER
I20260806 21:15:47.090579 14216 ts_tablet_manager.cc:1428] T 714cecfd274f4a42bbbb5f9a405751d0 P f290d0eb984f4c6d8cef0e5c64c1380d: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260806 21:15:47.090721 14215 consensus_queue.cc:260] T 714cecfd274f4a42bbbb5f9a405751d0 P 7ec42ff6d2874d2c9cfd308ea7139a35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec42ff6d2874d2c9cfd308ea7139a35" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 34739 } } peers { permanent_uuid: "f290d0eb984f4c6d8cef0e5c64c1380d" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 32851 } } peers { permanent_uuid: "e879b1bdf4614d5aa43258462e7a0c40" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 46735 } }
I20260806 21:15:47.091015 13803 heartbeater.cc:503] Master 127.16.178.62:38065 was elected leader, sending a full tablet report...
I20260806 21:15:47.091078 13993 heartbeater.cc:503] Master 127.16.178.62:38065 was elected leader, sending a full tablet report...
I20260806 21:15:47.091367 14195 heartbeater.cc:503] Master 127.16.178.62:38065 was elected leader, sending a full tablet report...
I20260806 21:15:47.091655 14215 ts_tablet_manager.cc:1428] T 714cecfd274f4a42bbbb5f9a405751d0 P 7ec42ff6d2874d2c9cfd308ea7139a35: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260806 21:15:47.092137 14214 ts_tablet_manager.cc:1428] T 714cecfd274f4a42bbbb5f9a405751d0 P e879b1bdf4614d5aa43258462e7a0c40: Time spent starting tablet: real 0.007s user 0.004s sys 0.003s
I20260806 21:15:47.094833 14224 raft_consensus.cc:482] T 714cecfd274f4a42bbbb5f9a405751d0 P 7ec42ff6d2874d2c9cfd308ea7139a35 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:15:47.095000 14224 raft_consensus.cc:504] T 714cecfd274f4a42bbbb5f9a405751d0 P 7ec42ff6d2874d2c9cfd308ea7139a35 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec42ff6d2874d2c9cfd308ea7139a35" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 34739 } } peers { permanent_uuid: "f290d0eb984f4c6d8cef0e5c64c1380d" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 32851 } } peers { permanent_uuid: "e879b1bdf4614d5aa43258462e7a0c40" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 46735 } }
I20260806 21:15:47.096697 14224 leader_election.cc:288] T 714cecfd274f4a42bbbb5f9a405751d0 P 7ec42ff6d2874d2c9cfd308ea7139a35 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f290d0eb984f4c6d8cef0e5c64c1380d (127.16.178.2:32851), e879b1bdf4614d5aa43258462e7a0c40 (127.16.178.1:46735)
I20260806 21:15:47.101154 14225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:47.097125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:46735 (local address 127.16.178.3:47303)
0806 21:15:47.097462 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:47.097475 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:47.097563 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:47.097761 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:47.097763 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:47.097779 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:15:47.097884 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:47.097900 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:47.098675 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:47.098678 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:47.100125 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:47.100137 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:47.100745 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:47.100748 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:47.100824 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:47.100846 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 21:15:47.100917 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 21:15:47.100990 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":70,"thread_start_us":137,"threads_started":1}
I20260806 21:15:47.101140 14212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:47.097257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:47303 (local address 127.16.178.1:46735)
0806 21:15:47.097371 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:47.097376 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:47.097547 (+ 171us) server_negotiation.cc:407] Connection header received
0806 21:15:47.097647 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:47.097649 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:47.097688 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:47.097738 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:15:47.098002 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:47.098556 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:47.100246 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:47.100645 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:47.100744 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:47.100784 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:47.100954 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 21:15:47.101044 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 21:15:47.101740 14197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:47.096843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.2:32851 (local address 127.16.178.3:40941)
0806 21:15:47.097051 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:47.097065 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:47.097151 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:47.097596 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:47.097599 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:47.097613 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:15:47.097725 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:47.097732 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:47.098639 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:47.098643 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:47.100127 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:47.100137 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:47.101392 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:47.101397 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:47.101495 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:47.101518 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 21:15:47.101596 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 21:15:47.101654 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":112}
I20260806 21:15:47.101836 14213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:47.096915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:40941 (local address 127.16.178.2:32851)
0806 21:15:47.097292 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:47.097297 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:47.097310 (+ 13us) server_negotiation.cc:407] Connection header received
0806 21:15:47.097354 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:47.097357 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:47.097416 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:47.097483 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:15:47.097845 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:47.098525 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:47.100836 (+ 2311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:47.101240 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:47.101398 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:47.101439 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:47.101627 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 21:15:47.101698 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309}
I20260806 21:15:47.101788 13741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "714cecfd274f4a42bbbb5f9a405751d0" candidate_uuid: "7ec42ff6d2874d2c9cfd308ea7139a35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e879b1bdf4614d5aa43258462e7a0c40" is_pre_election: true
I20260806 21:15:47.102123 13741 raft_consensus.cc:2454] T 714cecfd274f4a42bbbb5f9a405751d0 P e879b1bdf4614d5aa43258462e7a0c40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ec42ff6d2874d2c9cfd308ea7139a35 in term 0.
I20260806 21:15:47.102365 13926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "714cecfd274f4a42bbbb5f9a405751d0" candidate_uuid: "7ec42ff6d2874d2c9cfd308ea7139a35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f290d0eb984f4c6d8cef0e5c64c1380d" is_pre_election: true
I20260806 21:15:47.102618 14020 leader_election.cc:302] T 714cecfd274f4a42bbbb5f9a405751d0 P 7ec42ff6d2874d2c9cfd308ea7139a35 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ec42ff6d2874d2c9cfd308ea7139a35, e879b1bdf4614d5aa43258462e7a0c40; no voters:
I20260806 21:15:47.102643 13926 raft_consensus.cc:2454] T 714cecfd274f4a42bbbb5f9a405751d0 P f290d0eb984f4c6d8cef0e5c64c1380d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ec42ff6d2874d2c9cfd308ea7139a35 in term 0.
I20260806 21:15:47.102870 14224 raft_consensus.cc:2781] T 714cecfd274f4a42bbbb5f9a405751d0 P 7ec42ff6d2874d2c9cfd308ea7139a35 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:15:47.102972 14224 raft_consensus.cc:482] T 714cecfd274f4a42bbbb5f9a405751d0 P 7ec42ff6d2874d2c9cfd308ea7139a35 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:15:47.103039 14224 raft_consensus.cc:3037] T 714cecfd274f4a42bbbb5f9a405751d0 P 7ec42ff6d2874d2c9cfd308ea7139a35 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:15:47.104023 14224 raft_consensus.cc:504] T 714cecfd274f4a42bbbb5f9a405751d0 P 7ec42ff6d2874d2c9cfd308ea7139a35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec42ff6d2874d2c9cfd308ea7139a35" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 34739 } } peers { permanent_uuid: "f290d0eb984f4c6d8cef0e5c64c1380d" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 32851 } } peers { permanent_uuid: "e879b1bdf4614d5aa43258462e7a0c40" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 46735 } }
I20260806 21:15:47.104401 14224 leader_election.cc:288] T 714cecfd274f4a42bbbb5f9a405751d0 P 7ec42ff6d2874d2c9cfd308ea7139a35 [CANDIDATE]: Term 1 election: Requested vote from peers f290d0eb984f4c6d8cef0e5c64c1380d (127.16.178.2:32851), e879b1bdf4614d5aa43258462e7a0c40 (127.16.178.1:46735)
I20260806 21:15:47.104735 13926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "714cecfd274f4a42bbbb5f9a405751d0" candidate_uuid: "7ec42ff6d2874d2c9cfd308ea7139a35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f290d0eb984f4c6d8cef0e5c64c1380d"
I20260806 21:15:47.104800 13741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "714cecfd274f4a42bbbb5f9a405751d0" candidate_uuid: "7ec42ff6d2874d2c9cfd308ea7139a35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e879b1bdf4614d5aa43258462e7a0c40"
I20260806 21:15:47.104847 13926 raft_consensus.cc:3037] T 714cecfd274f4a42bbbb5f9a405751d0 P f290d0eb984f4c6d8cef0e5c64c1380d [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:15:47.104928 13741 raft_consensus.cc:3037] T 714cecfd274f4a42bbbb5f9a405751d0 P e879b1bdf4614d5aa43258462e7a0c40 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:15:47.105854 13926 raft_consensus.cc:2454] T 714cecfd274f4a42bbbb5f9a405751d0 P f290d0eb984f4c6d8cef0e5c64c1380d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ec42ff6d2874d2c9cfd308ea7139a35 in term 1.
I20260806 21:15:47.105998 13741 raft_consensus.cc:2454] T 714cecfd274f4a42bbbb5f9a405751d0 P e879b1bdf4614d5aa43258462e7a0c40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ec42ff6d2874d2c9cfd308ea7139a35 in term 1.
I20260806 21:15:47.106181 14020 leader_election.cc:302] T 714cecfd274f4a42bbbb5f9a405751d0 P 7ec42ff6d2874d2c9cfd308ea7139a35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ec42ff6d2874d2c9cfd308ea7139a35, f290d0eb984f4c6d8cef0e5c64c1380d; no voters:
I20260806 21:15:47.106465 14224 raft_consensus.cc:2781] T 714cecfd274f4a42bbbb5f9a405751d0 P 7ec42ff6d2874d2c9cfd308ea7139a35 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:15:47.106691 14224 raft_consensus.cc:686] T 714cecfd274f4a42bbbb5f9a405751d0 P 7ec42ff6d2874d2c9cfd308ea7139a35 [term 1 LEADER]: Becoming Leader. State: Replica: 7ec42ff6d2874d2c9cfd308ea7139a35, State: Running, Role: LEADER
I20260806 21:15:47.107023 14224 consensus_queue.cc:237] T 714cecfd274f4a42bbbb5f9a405751d0 P 7ec42ff6d2874d2c9cfd308ea7139a35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec42ff6d2874d2c9cfd308ea7139a35" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 34739 } } peers { permanent_uuid: "f290d0eb984f4c6d8cef0e5c64c1380d" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 32851 } } peers { permanent_uuid: "e879b1bdf4614d5aa43258462e7a0c40" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 46735 } }
W20260806 21:15:47.108669 13810 tablet.cc:2369] T 714cecfd274f4a42bbbb5f9a405751d0 P e879b1bdf4614d5aa43258462e7a0c40: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 21:15:47.110175 13533 catalog_manager.cc:5495] T 714cecfd274f4a42bbbb5f9a405751d0 P 7ec42ff6d2874d2c9cfd308ea7139a35 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ec42ff6d2874d2c9cfd308ea7139a35 (127.16.178.3). New cstate: current_term: 1 leader_uuid: "7ec42ff6d2874d2c9cfd308ea7139a35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec42ff6d2874d2c9cfd308ea7139a35" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 34739 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f290d0eb984f4c6d8cef0e5c64c1380d" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e879b1bdf4614d5aa43258462e7a0c40" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 46735 } health_report { overall_health: UNKNOWN } } }
I20260806 21:15:47.131747 14208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:47.128020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56590 (local address 127.16.178.3:34739)
0806 21:15:47.128193 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:47.128198 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:47.128213 (+ 15us) server_negotiation.cc:407] Connection header received
0806 21:15:47.128300 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:47.128302 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:47.128360 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:47.128454 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:15:47.128737 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:47.129238 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:47.130124 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:47.130336 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:47.130422 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:47.130775 (+ 353us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:15:47.131249 (+ 474us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:15:47.131407 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:47.131533 (+ 126us) server_negotiation.cc:299] Negotiation successful
0806 21:15:47.131576 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 21:15:47.138141 13741 raft_consensus.cc:1261] T 714cecfd274f4a42bbbb5f9a405751d0 P e879b1bdf4614d5aa43258462e7a0c40 [term 1 FOLLOWER]: Refusing update from remote peer 7ec42ff6d2874d2c9cfd308ea7139a35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:15:47.138140 13926 raft_consensus.cc:1261] T 714cecfd274f4a42bbbb5f9a405751d0 P f290d0eb984f4c6d8cef0e5c64c1380d [term 1 FOLLOWER]: Refusing update from remote peer 7ec42ff6d2874d2c9cfd308ea7139a35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:15:47.138895 14226 consensus_queue.cc:1035] T 714cecfd274f4a42bbbb5f9a405751d0 P 7ec42ff6d2874d2c9cfd308ea7139a35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e879b1bdf4614d5aa43258462e7a0c40" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 46735 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:15:47.139077 14224 consensus_queue.cc:1035] T 714cecfd274f4a42bbbb5f9a405751d0 P 7ec42ff6d2874d2c9cfd308ea7139a35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f290d0eb984f4c6d8cef0e5c64c1380d" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:15:47.152661 14232 mvcc.cc:205] Tried to move back new op lower bound from 7315664679470002176 to 7315664679353020416. Current Snapshot: MvccSnapshot[applied={T|T < 7315664679470002176}]
I20260806 21:15:47.154289 14234 mvcc.cc:205] Tried to move back new op lower bound from 7315664679470002176 to 7315664679353020416. Current Snapshot: MvccSnapshot[applied={T|T < 7315664679470002176}]
I20260806 21:15:47.162575 13533 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51502:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:15:47.162781 13533 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51502:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:15:47.165740 13533 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f2470ddce0594bf4a4d5f7fa70e19863: Sending DeleteTablet for 3 replicas of tablet 714cecfd274f4a42bbbb5f9a405751d0
I20260806 21:15:47.166777 13901 tablet_service.cc:1514] Processing DeleteTablet for tablet 714cecfd274f4a42bbbb5f9a405751d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:15:47 UTC) from {username='slave'} at 127.0.0.1:49006
I20260806 21:15:47.167338 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 13619
I20260806 21:15:47.167378 13716 tablet_service.cc:1514] Processing DeleteTablet for tablet 714cecfd274f4a42bbbb5f9a405751d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:15:47 UTC) from {username='slave'} at 127.0.0.1:45188
I20260806 21:15:47.167941 14091 tablet_service.cc:1514] Processing DeleteTablet for tablet 714cecfd274f4a42bbbb5f9a405751d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:15:47 UTC) from {username='slave'} at 127.0.0.1:56578
I20260806 21:15:47.170711 14253 tablet_replica.cc:330] T 714cecfd274f4a42bbbb5f9a405751d0 P 7ec42ff6d2874d2c9cfd308ea7139a35: stopping tablet replica
I20260806 21:15:47.171162 14253 raft_consensus.cc:2229] T 714cecfd274f4a42bbbb5f9a405751d0 P 7ec42ff6d2874d2c9cfd308ea7139a35 [term 1 LEADER]: Raft consensus shutting down.
I20260806 21:15:47.171653 14253 raft_consensus.cc:2258] T 714cecfd274f4a42bbbb5f9a405751d0 P 7ec42ff6d2874d2c9cfd308ea7139a35 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:15:47.174506 14251 tablet_replica.cc:330] T 714cecfd274f4a42bbbb5f9a405751d0 P f290d0eb984f4c6d8cef0e5c64c1380d: stopping tablet replica
I20260806 21:15:47.174858 14251 raft_consensus.cc:2229] T 714cecfd274f4a42bbbb5f9a405751d0 P f290d0eb984f4c6d8cef0e5c64c1380d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:15:47.175233 14251 raft_consensus.cc:2258] T 714cecfd274f4a42bbbb5f9a405751d0 P f290d0eb984f4c6d8cef0e5c64c1380d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:15:47.175477 14253 ts_tablet_manager.cc:1905] T 714cecfd274f4a42bbbb5f9a405751d0 P 7ec42ff6d2874d2c9cfd308ea7139a35: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:15:47.178658 14253 ts_tablet_manager.cc:1918] T 714cecfd274f4a42bbbb5f9a405751d0 P 7ec42ff6d2874d2c9cfd308ea7139a35: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:15:47.178845 14253 log.cc:1196] T 714cecfd274f4a42bbbb5f9a405751d0 P 7ec42ff6d2874d2c9cfd308ea7139a35: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786050686797304-17096-0/minicluster-data/ts-2/wal/wals/714cecfd274f4a42bbbb5f9a405751d0
I20260806 21:15:47.179268 14253 ts_tablet_manager.cc:1939] T 714cecfd274f4a42bbbb5f9a405751d0 P 7ec42ff6d2874d2c9cfd308ea7139a35: Deleting consensus metadata
I20260806 21:15:47.180945 13513 catalog_manager.cc:4841] TS 7ec42ff6d2874d2c9cfd308ea7139a35 (127.16.178.3:34739): tablet 714cecfd274f4a42bbbb5f9a405751d0 (table test-table [id=9caab356bf634e6bb4e45fcda3633a78]) successfully deleted
W20260806 21:15:47.184703 13513 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.178.1:46735 (error 108)
I20260806 21:15:47.185894 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 13814
I20260806 21:15:47.186389 14211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:47.186030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:46735 (local address 127.0.0.1:45190)
0806 21:15:47.186228 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:47.186295 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.1:46735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
W20260806 21:15:47.191020 13513 catalog_manager.cc:4584] TS e879b1bdf4614d5aa43258462e7a0c40 (127.16.178.1:46735): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 714cecfd274f4a42bbbb5f9a405751d0: Network error: Client connection negotiation failed: client connection to 127.16.178.1:46735: connect: Connection refused (error 111)
I20260806 21:15:47.203289 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 13996
I20260806 21:15:47.203552 14211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:47.203225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.2:32851 (local address 127.0.0.1:49018)
0806 21:15:47.203387 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:47.203450 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.2:32851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260806 21:15:47.225342 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 13487
2026-08-06T21:15:47Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1118 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:15:47 dist-test-slave-lgr5 krb5kdc[14258](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:15:47 dist-test-slave-lgr5 krb5kdc[14258](info): set up 2 sockets
Aug 06 21:15:47 dist-test-slave-lgr5 krb5kdc[14258](info): commencing operation
krb5kdc: starting...
W20260806 21:15:48.375187 17096 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Aug 06 21:15:48 dist-test-slave-lgr5 krb5kdc[14258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:15:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:15:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.178.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
I20260806 21:15:48.508960 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:46001
--webserver_interface=127.16.178.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:43069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:46001
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:15:48.626041 14279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:15:48.626327 14279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:15:48.626365 14279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:15:48.630431 14279 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:15:48.630509 14279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:15:48.630529 14279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:15:48.630544 14279 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:15:48.630559 14279 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:15:48.636303 14279 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:43069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:46001
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:46001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.178.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14279
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:15:48.637458 14279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:15:48.638504 14279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:15:48.646916 14292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:48.646916 14291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:48.646937 14295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:48.647250 14279 server_base.cc:1034] running on GCE node
I20260806 21:15:48.647779 14279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:15:48.648926 14279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:15:48.650087 14279 hybrid_clock.cc:648] HybridClock initialized: now 1786050948650066 us; error 37 us; skew 500 ppm
Aug 06 21:15:48 dist-test-slave-lgr5 krb5kdc[14258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050948, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:15:48.652760 14279 init.cc:377] Logged in from keytab as kudu/127.16.178.62@KRBTEST.COM (short username kudu)
I20260806 21:15:48.653609 14279 webserver.cc:460] Webserver started at http://127.16.178.62:38561/ using document root <none> and password file <none>
I20260806 21:15:48.654129 14279 fs_manager.cc:359] Metadata directory not provided
I20260806 21:15:48.654186 14279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:15:48.654352 14279 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:15:48.656476 14279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "e0e45ebb880d4a2bb08cbac758fd60ea"
format_stamp: "Formatted at 2026-08-06 21:15:48 on dist-test-slave-lgr5"
server_key: "45dffd96176551f109bf01c72017fd5b"
server_key_iv: "c0b3a20f2aea099a8c8436ac923d22ca"
server_key_version: "encryptionkey@0"
I20260806 21:15:48.657003 14279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "e0e45ebb880d4a2bb08cbac758fd60ea"
format_stamp: "Formatted at 2026-08-06 21:15:48 on dist-test-slave-lgr5"
server_key: "45dffd96176551f109bf01c72017fd5b"
server_key_iv: "c0b3a20f2aea099a8c8436ac923d22ca"
server_key_version: "encryptionkey@0"
I20260806 21:15:48.660683 14279 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 21:15:48.663208 14303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:48.664598 14279 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:15:48.664733 14279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "e0e45ebb880d4a2bb08cbac758fd60ea"
format_stamp: "Formatted at 2026-08-06 21:15:48 on dist-test-slave-lgr5"
server_key: "45dffd96176551f109bf01c72017fd5b"
server_key_iv: "c0b3a20f2aea099a8c8436ac923d22ca"
server_key_version: "encryptionkey@0"
I20260806 21:15:48.664858 14279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:15:48.691428 14279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:15:48.695222 14279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:15:48.695439 14279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:15:48.702840 14279 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.62:46001
I20260806 21:15:48.702926 14379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.62:46001 every 8 connection(s)
I20260806 21:15:48.703876 14279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:15:48.706565 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 14279
I20260806 21:15:48.706717 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:15:48.707036 14380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:15:48.707055 17096 external_mini_cluster.cc:1428] Setting key 6ff5d7bc3d4f7bdb23952bed0a3dd771
I20260806 21:15:48.714182 14380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0e45ebb880d4a2bb08cbac758fd60ea: Bootstrap starting.
Aug 06 21:15:48 dist-test-slave-lgr5 krb5kdc[14258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:15:48.717047 14380 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0e45ebb880d4a2bb08cbac758fd60ea: Neither blocks nor log segments found. Creating new log.
I20260806 21:15:48.717917 14380 log.cc:824] T 00000000000000000000000000000000 P e0e45ebb880d4a2bb08cbac758fd60ea: Log is configured to *not* fsync() on all Append() calls
I20260806 21:15:48.720242 14380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0e45ebb880d4a2bb08cbac758fd60ea: No bootstrap required, opened a new log
I20260806 21:15:48.722131 14384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:48.708493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34268 (local address 127.16.178.62:46001)
0806 21:15:48.708941 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:48.708957 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:48.708995 (+ 38us) server_negotiation.cc:407] Connection header received
0806 21:15:48.709906 (+ 911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:48.709936 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:48.710278 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:48.710571 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:15:48.711477 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:48.712537 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:48.713664 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:48.714139 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:48.714254 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:48.717064 (+ 2810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:15:48.717069 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:15:48.717083 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:15:48.717112 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:15:48.719007 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:48.719478 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:48.719483 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:48.719487 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:48.719643 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:48.719934 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:48.719936 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:48.719938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:48.720427 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:15:48.720602 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:48.720858 (+ 256us) server_negotiation.cc:299] Negotiation successful
0806 21:15:48.721137 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":122,"threads_started":1}
I20260806 21:15:48.723675 14380 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0e45ebb880d4a2bb08cbac758fd60ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0e45ebb880d4a2bb08cbac758fd60ea" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 46001 } }
I20260806 21:15:48.723955 14380 raft_consensus.cc:374] T 00000000000000000000000000000000 P e0e45ebb880d4a2bb08cbac758fd60ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:15:48.724014 14380 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0e45ebb880d4a2bb08cbac758fd60ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0e45ebb880d4a2bb08cbac758fd60ea, State: Initialized, Role: FOLLOWER
I20260806 21:15:48.724470 14380 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0e45ebb880d4a2bb08cbac758fd60ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0e45ebb880d4a2bb08cbac758fd60ea" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 46001 } }
I20260806 21:15:48.724609 14380 raft_consensus.cc:388] T 00000000000000000000000000000000 P e0e45ebb880d4a2bb08cbac758fd60ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:15:48.724673 14380 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0e45ebb880d4a2bb08cbac758fd60ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:15:48.724771 14380 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0e45ebb880d4a2bb08cbac758fd60ea [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:15:48.725701 14380 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0e45ebb880d4a2bb08cbac758fd60ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0e45ebb880d4a2bb08cbac758fd60ea" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 46001 } }
I20260806 21:15:48.726016 14380 leader_election.cc:302] T 00000000000000000000000000000000 P e0e45ebb880d4a2bb08cbac758fd60ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e0e45ebb880d4a2bb08cbac758fd60ea; no voters:
I20260806 21:15:48.726351 14380 leader_election.cc:288] T 00000000000000000000000000000000 P e0e45ebb880d4a2bb08cbac758fd60ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:15:48.726437 14389 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0e45ebb880d4a2bb08cbac758fd60ea [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:15:48.727038 14389 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0e45ebb880d4a2bb08cbac758fd60ea [term 1 LEADER]: Becoming Leader. State: Replica: e0e45ebb880d4a2bb08cbac758fd60ea, State: Running, Role: LEADER
I20260806 21:15:48.727376 14389 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0e45ebb880d4a2bb08cbac758fd60ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0e45ebb880d4a2bb08cbac758fd60ea" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 46001 } }
I20260806 21:15:48.727514 14380 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0e45ebb880d4a2bb08cbac758fd60ea [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:15:48.729014 14394 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0e45ebb880d4a2bb08cbac758fd60ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0e45ebb880d4a2bb08cbac758fd60ea. Latest consensus state: current_term: 1 leader_uuid: "e0e45ebb880d4a2bb08cbac758fd60ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0e45ebb880d4a2bb08cbac758fd60ea" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 46001 } } }
I20260806 21:15:48.729070 14391 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0e45ebb880d4a2bb08cbac758fd60ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e0e45ebb880d4a2bb08cbac758fd60ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0e45ebb880d4a2bb08cbac758fd60ea" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 46001 } } }
I20260806 21:15:48.729188 14391 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0e45ebb880d4a2bb08cbac758fd60ea [sys.catalog]: This master's current role is: LEADER
I20260806 21:15:48.729139 14394 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0e45ebb880d4a2bb08cbac758fd60ea [sys.catalog]: This master's current role is: LEADER
I20260806 21:15:48.729580 14411 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:15:48.732856 14411 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:15:48.738691 14411 catalog_manager.cc:1347] Generated new cluster ID: cc5c09e5e53e4f738117357b6f74e546
I20260806 21:15:48.738770 14411 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:15:48.756412 14411 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:15:48.757454 14411 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:15:48.761497 14411 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e0e45ebb880d4a2bb08cbac758fd60ea: Generated new TSK 0
I20260806 21:15:48.762259 14411 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.178.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab.
Entry for principal kudu/127.16.178.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab.
I20260806 21:15:48.799576 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.1:0
--local_ip_for_outbound_sockets=127.16.178.1
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:46001
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab
--principal=kudu/127.16.178.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:43069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:15:48.947988 14423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:15:48.948308 14423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:15:48.948352 14423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:15:48.952497 14423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:15:48.952574 14423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:15:48.952636 14423 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.1
I20260806 21:15:48.958091 14423 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:43069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.1.keytab
--principal=kudu/127.16.178.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:46001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14423
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.1
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:15:48.959254 14423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:15:48.960355 14423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 21:15:48.970911 14423 server_base.cc:1034] running on GCE node
W20260806 21:15:48.970836 14428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:48.970839 14430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:48.970836 14432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:48.971696 14423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:15:48.972536 14423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:15:48.973737 14423 hybrid_clock.cc:648] HybridClock initialized: now 1786050948973716 us; error 36 us; skew 500 ppm
Aug 06 21:15:48 dist-test-slave-lgr5 krb5kdc[14258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050948, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:15:48.976929 14423 init.cc:377] Logged in from keytab as kudu/127.16.178.1@KRBTEST.COM (short username kudu)
I20260806 21:15:48.978055 14423 webserver.cc:460] Webserver started at http://127.16.178.1:35603/ using document root <none> and password file <none>
I20260806 21:15:48.978716 14423 fs_manager.cc:359] Metadata directory not provided
I20260806 21:15:48.978786 14423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:15:48.979005 14423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:15:48.981626 14423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-0/data/instance:
uuid: "31f2e75cea374d9ebbed7a9da90c2c23"
format_stamp: "Formatted at 2026-08-06 21:15:48 on dist-test-slave-lgr5"
server_key: "6d9d7a535284a6842b90d8fb0c0efac2"
server_key_iv: "17a767ab19d687635b95416c5e99e64e"
server_key_version: "encryptionkey@0"
I20260806 21:15:48.982293 14423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance:
uuid: "31f2e75cea374d9ebbed7a9da90c2c23"
format_stamp: "Formatted at 2026-08-06 21:15:48 on dist-test-slave-lgr5"
server_key: "6d9d7a535284a6842b90d8fb0c0efac2"
server_key_iv: "17a767ab19d687635b95416c5e99e64e"
server_key_version: "encryptionkey@0"
I20260806 21:15:48.986904 14423 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 21:15:48.990011 14441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:48.991572 14423 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 21:15:48.991722 14423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-0/wal
uuid: "31f2e75cea374d9ebbed7a9da90c2c23"
format_stamp: "Formatted at 2026-08-06 21:15:48 on dist-test-slave-lgr5"
server_key: "6d9d7a535284a6842b90d8fb0c0efac2"
server_key_iv: "17a767ab19d687635b95416c5e99e64e"
server_key_version: "encryptionkey@0"
I20260806 21:15:48.991871 14423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:15:49.001957 14423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:15:49.005815 14423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:15:49.006007 14423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:15:49.006728 14423 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:15:49.007727 14423 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:15:49.007794 14423 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:49.007841 14423 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:15:49.007855 14423 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:49.017195 14423 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.1:34079
I20260806 21:15:49.017223 14615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.1:34079 every 8 connection(s)
I20260806 21:15:49.018178 14423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
I20260806 21:15:49.018956 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 14423
I20260806 21:15:49.019097 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance
I20260806 21:15:49.019450 17096 external_mini_cluster.cc:1428] Setting key 47b7507978ae8cae01baf2d12624d0e8
WARNING: no policy specified for kudu/127.16.178.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.2@KRBTEST.COM" created.
Aug 06 21:15:49 dist-test-slave-lgr5 krb5kdc[14258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050948, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.1@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:15:49.038695 14384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:49.020370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:39813 (local address 127.16.178.62:46001)
0806 21:15:49.020505 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:49.020509 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:49.023972 (+ 3463us) server_negotiation.cc:407] Connection header received
0806 21:15:49.025427 (+ 1455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:49.025430 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:49.025501 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:49.025590 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:15:49.027063 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:49.027916 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:49.029323 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:49.029716 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:49.029862 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:49.034781 (+ 4919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:15:49.034786 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:15:49.034789 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:15:49.034819 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:15:49.036438 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:49.037033 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:49.037037 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:49.037039 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:49.037136 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:49.037510 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:49.037513 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:49.037515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:49.037717 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:15:49.037807 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:49.038367 (+ 560us) server_negotiation.cc:299] Negotiation successful
0806 21:15:49.038519 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 21:15:49.039534 14618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:49.021029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:46001 (local address 127.16.178.1:39813)
0806 21:15:49.023744 (+ 2715us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:49.023810 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:49.025171 (+ 1361us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:49.025832 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:49.025840 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:49.026250 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:15:49.026835 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:49.026851 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:49.028124 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:49.028128 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:49.029183 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:49.029194 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:49.029897 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:49.029900 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:49.030079 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:49.030710 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:15:49.030727 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:15:49.034568 (+ 3841us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:15:49.036603 (+ 2035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:15:49.036610 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:15:49.036628 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:15:49.036911 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:15:49.037245 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:15:49.037249 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:15:49.037251 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:15:49.037402 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:15:49.037822 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:15:49.037831 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:15:49.038116 (+ 285us) client_negotiation.cc:769] Sending connection context
0806 21:15:49.038291 (+ 175us) client_negotiation.cc:240] Negotiation successful
0806 21:15:49.038657 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2434,"thread_start_us":167,"threads_started":1}
I20260806 21:15:49.041008 14616 heartbeater.cc:347] Connected to a master server at 127.16.178.62:46001
I20260806 21:15:49.041268 14616 heartbeater.cc:464] Registering TS with master...
I20260806 21:15:49.041797 14616 heartbeater.cc:511] Master 127.16.178.62:46001 requested a full tablet report, sending...
I20260806 21:15:49.043637 14328 ts_manager.cc:194] Registered new tserver with Master: 31f2e75cea374d9ebbed7a9da90c2c23 (127.16.178.1:34079)
I20260806 21:15:49.045090 14328 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.1@KRBTEST.COM'} at 127.16.178.1:39813
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab.
Entry for principal kudu/127.16.178.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab.
I20260806 21:15:49.051008 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.2:0
--local_ip_for_outbound_sockets=127.16.178.2
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:46001
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab
--principal=kudu/127.16.178.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:43069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:15:49.171983 14623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:15:49.172314 14623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:15:49.172360 14623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:15:49.176620 14623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:15:49.176694 14623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:15:49.176764 14623 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.2
I20260806 21:15:49.182263 14623 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:43069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.2.keytab
--principal=kudu/127.16.178.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:46001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.2
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:15:49.183406 14623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:15:49.184525 14623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:15:49.194468 14634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:49.194603 14623 server_base.cc:1034] running on GCE node
W20260806 21:15:49.194468 14635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:49.194785 14637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:49.195328 14623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:15:49.198110 14623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:15:49.199277 14623 hybrid_clock.cc:648] HybridClock initialized: now 1786050949199242 us; error 57 us; skew 500 ppm
Aug 06 21:15:49 dist-test-slave-lgr5 krb5kdc[14258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050949, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:15:49.202093 14623 init.cc:377] Logged in from keytab as kudu/127.16.178.2@KRBTEST.COM (short username kudu)
I20260806 21:15:49.203120 14623 webserver.cc:460] Webserver started at http://127.16.178.2:37285/ using document root <none> and password file <none>
I20260806 21:15:49.203738 14623 fs_manager.cc:359] Metadata directory not provided
I20260806 21:15:49.203821 14623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:15:49.204200 14623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:15:49.206610 14623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-1/data/instance:
uuid: "3ce02587d52445a1aaa86fa4a33b6879"
format_stamp: "Formatted at 2026-08-06 21:15:49 on dist-test-slave-lgr5"
server_key: "69086f503625e1dc79f95f0316c8ddbf"
server_key_iv: "b2c2f02bc36d6ff74eb8ed0089735059"
server_key_version: "encryptionkey@0"
I20260806 21:15:49.207343 14623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance:
uuid: "3ce02587d52445a1aaa86fa4a33b6879"
format_stamp: "Formatted at 2026-08-06 21:15:49 on dist-test-slave-lgr5"
server_key: "69086f503625e1dc79f95f0316c8ddbf"
server_key_iv: "b2c2f02bc36d6ff74eb8ed0089735059"
server_key_version: "encryptionkey@0"
I20260806 21:15:49.211961 14623 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 21:15:49.214825 14647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:49.216272 14623 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:15:49.216437 14623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-1/wal
uuid: "3ce02587d52445a1aaa86fa4a33b6879"
format_stamp: "Formatted at 2026-08-06 21:15:49 on dist-test-slave-lgr5"
server_key: "69086f503625e1dc79f95f0316c8ddbf"
server_key_iv: "b2c2f02bc36d6ff74eb8ed0089735059"
server_key_version: "encryptionkey@0"
I20260806 21:15:49.216557 14623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:15:49.241215 14623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:15:49.244297 14623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:15:49.244477 14623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:15:49.245088 14623 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:15:49.245926 14623 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:15:49.245975 14623 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:49.246037 14623 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:15:49.246062 14623 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:49.254748 14623 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.2:44859
I20260806 21:15:49.254778 14818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.2:44859 every 8 connection(s)
I20260806 21:15:49.255718 14623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
I20260806 21:15:49.259197 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 14623
I20260806 21:15:49.259302 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance
I20260806 21:15:49.259583 17096 external_mini_cluster.cc:1428] Setting key 4322457a1c0fcbf653d375293ce2f795
Aug 06 21:15:49 dist-test-slave-lgr5 krb5kdc[14258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050949, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.2@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:15:49.270224 14384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:49.257660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:37575 (local address 127.16.178.62:46001)
0806 21:15:49.257818 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:49.257822 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:49.258843 (+ 1021us) server_negotiation.cc:407] Connection header received
0806 21:15:49.259815 (+ 972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:49.259818 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:49.259877 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:49.259969 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:15:49.261463 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:49.262168 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:49.263447 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:49.263681 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:49.263764 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:49.266436 (+ 2672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:15:49.266439 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:15:49.266441 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:15:49.266464 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:15:49.268020 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:49.268554 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:49.268558 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:49.268560 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:49.268652 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:49.268990 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:49.268993 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:49.268995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:49.269258 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:15:49.269360 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:49.269908 (+ 548us) server_negotiation.cc:299] Negotiation successful
0806 21:15:49.270046 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
WARNING: no policy specified for kudu/127.16.178.3@KRBTEST.COM; defaulting to no policy
I20260806 21:15:49.271026 14824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:49.258019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:46001 (local address 127.16.178.2:37575)
0806 21:15:49.258698 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:49.258739 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:49.259591 (+ 852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:49.260190 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:49.260197 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:49.260704 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:15:49.261317 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:49.261333 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:49.262315 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:49.262318 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:49.263319 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:49.263332 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:49.263829 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:49.263833 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:49.264031 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:49.264675 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:15:49.264691 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:15:49.266269 (+ 1578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:15:49.268166 (+ 1897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:15:49.268172 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:15:49.268186 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:15:49.268466 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:15:49.268733 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:15:49.268735 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:15:49.268737 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:15:49.268901 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:15:49.269357 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:15:49.269363 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:15:49.269660 (+ 297us) client_negotiation.cc:769] Sending connection context
0806 21:15:49.269822 (+ 162us) client_negotiation.cc:240] Negotiation successful
0806 21:15:49.270123 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":121,"threads_started":1}
I20260806 21:15:49.272279 14821 heartbeater.cc:347] Connected to a master server at 127.16.178.62:46001
I20260806 21:15:49.272567 14821 heartbeater.cc:464] Registering TS with master...
I20260806 21:15:49.273101 14821 heartbeater.cc:511] Master 127.16.178.62:46001 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.3@KRBTEST.COM" created.
I20260806 21:15:49.274323 14328 ts_manager.cc:194] Registered new tserver with Master: 3ce02587d52445a1aaa86fa4a33b6879 (127.16.178.2:44859)
I20260806 21:15:49.275096 14328 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.2@KRBTEST.COM'} at 127.16.178.2:37575
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab.
Entry for principal kudu/127.16.178.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab.
I20260806 21:15:49.291283 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.3:0
--local_ip_for_outbound_sockets=127.16.178.3
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:46001
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab
--principal=kudu/127.16.178.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:43069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:15:49.408440 14830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:15:49.408756 14830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:15:49.408818 14830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:15:49.413012 14830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:15:49.413091 14830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:15:49.413177 14830 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.3
I20260806 21:15:49.419092 14830 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:43069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.3.keytab
--principal=kudu/127.16.178.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:46001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14830
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.3
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:15:49.420452 14830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:15:49.421671 14830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 21:15:49.432271 14830 server_base.cc:1034] running on GCE node
W20260806 21:15:49.432197 14837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:49.432199 14840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:49.432233 14838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:49.432902 14830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:15:49.433554 14830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:15:49.434713 14830 hybrid_clock.cc:648] HybridClock initialized: now 1786050949434691 us; error 39 us; skew 500 ppm
Aug 06 21:15:49 dist-test-slave-lgr5 krb5kdc[14258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050949, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:15:49.437533 14830 init.cc:377] Logged in from keytab as kudu/127.16.178.3@KRBTEST.COM (short username kudu)
I20260806 21:15:49.438400 14830 webserver.cc:460] Webserver started at http://127.16.178.3:34109/ using document root <none> and password file <none>
I20260806 21:15:49.438979 14830 fs_manager.cc:359] Metadata directory not provided
I20260806 21:15:49.439049 14830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:15:49.439263 14830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:15:49.441355 14830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-2/data/instance:
uuid: "92e4e653051449fc88f176c965ad676f"
format_stamp: "Formatted at 2026-08-06 21:15:49 on dist-test-slave-lgr5"
server_key: "fdafb19b615ba726ce8df6df28d6d26f"
server_key_iv: "c40c9708ada50b314b85be411dd54cdb"
server_key_version: "encryptionkey@0"
I20260806 21:15:49.441898 14830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance:
uuid: "92e4e653051449fc88f176c965ad676f"
format_stamp: "Formatted at 2026-08-06 21:15:49 on dist-test-slave-lgr5"
server_key: "fdafb19b615ba726ce8df6df28d6d26f"
server_key_iv: "c40c9708ada50b314b85be411dd54cdb"
server_key_version: "encryptionkey@0"
I20260806 21:15:49.445506 14830 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 21:15:49.447789 14856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:49.448992 14830 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 21:15:49.449103 14830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-2/wal
uuid: "92e4e653051449fc88f176c965ad676f"
format_stamp: "Formatted at 2026-08-06 21:15:49 on dist-test-slave-lgr5"
server_key: "fdafb19b615ba726ce8df6df28d6d26f"
server_key_iv: "c40c9708ada50b314b85be411dd54cdb"
server_key_version: "encryptionkey@0"
I20260806 21:15:49.449194 14830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:15:49.478222 14830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:15:49.484514 14830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:15:49.484740 14830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:15:49.485385 14830 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:15:49.486532 14830 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:15:49.486591 14830 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:49.486676 14830 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:15:49.486706 14830 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:49.496855 14830 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.3:41801
I20260806 21:15:49.496912 15033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.3:41801 every 8 connection(s)
I20260806 21:15:49.497825 14830 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
I20260806 21:15:49.498998 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 14830
I20260806 21:15:49.499133 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance
I20260806 21:15:49.499424 17096 external_mini_cluster.cc:1428] Setting key d7859bb14b718d0ce4a7dcf502fcf845
Aug 06 21:15:49 dist-test-slave-lgr5 krb5kdc[14258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050949, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.3@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:15:49.513100 14384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:49.500133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:57017 (local address 127.16.178.62:46001)
0806 21:15:49.500287 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:49.500291 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:49.501172 (+ 881us) server_negotiation.cc:407] Connection header received
0806 21:15:49.502505 (+ 1333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:49.502508 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:49.502566 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:49.502649 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:15:49.503801 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:49.504686 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:49.505797 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:49.506146 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:49.506248 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:49.509117 (+ 2869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:15:49.509120 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:15:49.509122 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:15:49.509148 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:15:49.510433 (+ 1285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:49.511081 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:49.511084 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:49.511085 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:49.511164 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:49.511530 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:49.511532 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:49.511533 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:49.511710 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:15:49.511780 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:49.512627 (+ 847us) server_negotiation.cc:299] Negotiation successful
0806 21:15:49.512939 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 21:15:49.514024 15038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:49.500529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:46001 (local address 127.16.178.3:57017)
0806 21:15:49.501035 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:49.501079 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:49.502305 (+ 1226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:49.502813 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:49.502820 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:49.503172 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:15:49.503660 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:49.503675 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:49.504798 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:49.504801 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:49.505686 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:49.505695 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:49.506272 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:49.506274 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:49.506411 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:49.506989 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:15:49.507008 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:15:49.508914 (+ 1906us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:15:49.510552 (+ 1638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:15:49.510558 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:15:49.510576 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:15:49.510962 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:15:49.511246 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:15:49.511249 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:15:49.511251 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:15:49.511446 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:15:49.511803 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:15:49.511809 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:15:49.512205 (+ 396us) client_negotiation.cc:769] Sending connection context
0806 21:15:49.512464 (+ 259us) client_negotiation.cc:240] Negotiation successful
0806 21:15:49.512818 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":127,"threads_started":1}
I20260806 21:15:49.515398 15035 heartbeater.cc:347] Connected to a master server at 127.16.178.62:46001
I20260806 21:15:49.515654 15035 heartbeater.cc:464] Registering TS with master...
I20260806 21:15:49.516228 15035 heartbeater.cc:511] Master 127.16.178.62:46001 requested a full tablet report, sending...
I20260806 21:15:49.517490 14328 ts_manager.cc:194] Registered new tserver with Master: 92e4e653051449fc88f176c965ad676f (127.16.178.3:41801)
I20260806 21:15:49.518106 14328 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.178.3@KRBTEST.COM'} at 127.16.178.3:57017
I20260806 21:15:49.525589 17096 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 21:15:49.535655 14384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:49.527655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34270 (local address 127.16.178.62:46001)
0806 21:15:49.527780 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:49.527785 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:49.527965 (+ 180us) server_negotiation.cc:407] Connection header received
0806 21:15:49.528139 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:49.528142 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:49.528193 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:49.528257 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:15:49.529061 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:49.529595 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:49.530444 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:49.530695 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:49.530795 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:49.532413 (+ 1618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:15:49.532416 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:15:49.532418 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:15:49.532448 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:15:49.533813 (+ 1365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:49.534274 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:49.534278 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:49.534280 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:49.534368 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:49.534654 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:49.534658 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:49.534659 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:49.534919 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:15:49.535063 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:49.535326 (+ 263us) server_negotiation.cc:299] Negotiation successful
0806 21:15:49.535476 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 21:15:49.539112 14328 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34270:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:15:49.542385 14328 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:15:49.558612 15048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:49.551564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:34079 (local address 127.0.0.1:38936)
0806 21:15:49.551943 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:49.551994 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:49.552147 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:49.553862 (+ 1715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:49.553865 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:49.553894 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:15:49.554020 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:49.554027 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:49.555150 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:49.555154 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:49.556772 (+ 1618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:49.556787 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:49.557968 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:49.557971 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:49.558104 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:49.558167 (+ 63us) client_negotiation.cc:769] Sending connection context
0806 21:15:49.558257 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 21:15:49.558429 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":56,"spinlock_wait_cycles":768,"thread_start_us":111,"threads_started":1}
I20260806 21:15:49.558921 15055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:49.552918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38936 (local address 127.16.178.1:34079)
0806 21:15:49.553230 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:49.553236 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:49.553259 (+ 23us) server_negotiation.cc:407] Connection header received
0806 21:15:49.553335 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:49.553341 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:49.553706 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:49.554154 (+ 448us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:15:49.554262 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:49.555038 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:49.556940 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:49.557813 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:49.558417 (+ 604us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:49.558582 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:49.558643 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 21:15:49.558756 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":150,"threads_started":1}
I20260806 21:15:49.559335 15049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:49.551596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.2:44859 (local address 127.0.0.1:39254)
0806 21:15:49.552606 (+ 1010us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:49.552619 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:49.552706 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:49.553830 (+ 1124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:49.553833 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:49.553849 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:15:49.553945 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:49.553951 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:49.555343 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:49.555347 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:49.557568 (+ 2221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:49.557579 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:49.558957 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:49.558961 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:49.559080 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:49.559104 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 21:15:49.559159 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 21:15:49.559218 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":944,"mutex_wait_us":850,"thread_start_us":126,"threads_started":1}
I20260806 21:15:49.559337 15051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:49.551814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.3:41801 (local address 127.0.0.1:38724)
0806 21:15:49.552866 (+ 1052us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:49.552876 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:49.552960 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:49.553647 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:49.553653 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:49.553688 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:15:49.553835 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:49.553842 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:49.555343 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:49.555347 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:49.556747 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:49.556759 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:49.558046 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:49.558049 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:49.558173 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:49.558199 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 21:15:49.558274 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 21:15:49.559194 (+ 920us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1006,"mutex_wait_us":181,"thread_start_us":93,"threads_started":1}
I20260806 21:15:49.560398 15054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:49.552517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39254 (local address 127.16.178.2:44859)
0806 21:15:49.553197 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:49.553207 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:49.553227 (+ 20us) server_negotiation.cc:407] Connection header received
0806 21:15:49.553294 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:49.553299 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:49.553454 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:49.553584 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:15:49.554074 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:49.555215 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:49.557710 (+ 2495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:49.558521 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:49.558617 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:49.559856 (+ 1239us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:49.559908 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 21:15:49.560253 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":134,"threads_started":1}
I20260806 21:15:49.559948 15053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:49.552350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38724 (local address 127.16.178.3:41801)
0806 21:15:49.552751 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:49.552764 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:49.552966 (+ 202us) server_negotiation.cc:407] Connection header received
0806 21:15:49.553067 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:49.553073 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:49.553288 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:49.553437 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:15:49.553962 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:49.555186 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:49.556887 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:49.557715 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:49.557807 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:49.557974 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:49.559682 (+ 1708us) server_negotiation.cc:299] Negotiation successful
0806 21:15:49.559806 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":168,"threads_started":1}
I20260806 21:15:49.562695 14523 tablet_service.cc:1467] Processing CreateTablet for tablet 97ed5d4cab1e43ff9d69fa3e40b2e314 (DEFAULT_TABLE table=test-table [id=d41cbf84b2e0478c88440c1c35f3ef84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:15:49.564043 14523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97ed5d4cab1e43ff9d69fa3e40b2e314. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:15:49.563649 14721 tablet_service.cc:1467] Processing CreateTablet for tablet 97ed5d4cab1e43ff9d69fa3e40b2e314 (DEFAULT_TABLE table=test-table [id=d41cbf84b2e0478c88440c1c35f3ef84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:15:49.563814 14928 tablet_service.cc:1467] Processing CreateTablet for tablet 97ed5d4cab1e43ff9d69fa3e40b2e314 (DEFAULT_TABLE table=test-table [id=d41cbf84b2e0478c88440c1c35f3ef84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:15:49.564698 14721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97ed5d4cab1e43ff9d69fa3e40b2e314. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:15:49.564983 14928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97ed5d4cab1e43ff9d69fa3e40b2e314. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:15:49.572196 15056 tablet_bootstrap.cc:492] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 31f2e75cea374d9ebbed7a9da90c2c23: Bootstrap starting.
I20260806 21:15:49.572803 15057 tablet_bootstrap.cc:492] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 3ce02587d52445a1aaa86fa4a33b6879: Bootstrap starting.
I20260806 21:15:49.574747 15056 tablet_bootstrap.cc:654] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 31f2e75cea374d9ebbed7a9da90c2c23: Neither blocks nor log segments found. Creating new log.
I20260806 21:15:49.574790 15057 tablet_bootstrap.cc:654] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 3ce02587d52445a1aaa86fa4a33b6879: Neither blocks nor log segments found. Creating new log.
I20260806 21:15:49.574820 15058 tablet_bootstrap.cc:492] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 92e4e653051449fc88f176c965ad676f: Bootstrap starting.
I20260806 21:15:49.575716 15056 log.cc:824] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 31f2e75cea374d9ebbed7a9da90c2c23: Log is configured to *not* fsync() on all Append() calls
I20260806 21:15:49.576182 15057 log.cc:824] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 3ce02587d52445a1aaa86fa4a33b6879: Log is configured to *not* fsync() on all Append() calls
I20260806 21:15:49.577564 15058 tablet_bootstrap.cc:654] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 92e4e653051449fc88f176c965ad676f: Neither blocks nor log segments found. Creating new log.
I20260806 21:15:49.578730 15057 tablet_bootstrap.cc:492] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 3ce02587d52445a1aaa86fa4a33b6879: No bootstrap required, opened a new log
I20260806 21:15:49.578907 15058 log.cc:824] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 92e4e653051449fc88f176c965ad676f: Log is configured to *not* fsync() on all Append() calls
I20260806 21:15:49.578976 15057 ts_tablet_manager.cc:1397] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 3ce02587d52445a1aaa86fa4a33b6879: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 21:15:49.579836 15056 tablet_bootstrap.cc:492] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 31f2e75cea374d9ebbed7a9da90c2c23: No bootstrap required, opened a new log
I20260806 21:15:49.580060 15056 ts_tablet_manager.cc:1397] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 31f2e75cea374d9ebbed7a9da90c2c23: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.002s
I20260806 21:15:49.581089 15058 tablet_bootstrap.cc:492] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 92e4e653051449fc88f176c965ad676f: No bootstrap required, opened a new log
I20260806 21:15:49.581303 15058 ts_tablet_manager.cc:1397] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 92e4e653051449fc88f176c965ad676f: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 21:15:49.583084 15057 raft_consensus.cc:348] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 3ce02587d52445a1aaa86fa4a33b6879 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31f2e75cea374d9ebbed7a9da90c2c23" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 34079 } } peers { permanent_uuid: "3ce02587d52445a1aaa86fa4a33b6879" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 44859 } } peers { permanent_uuid: "92e4e653051449fc88f176c965ad676f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 41801 } }
I20260806 21:15:49.583429 15057 raft_consensus.cc:374] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 3ce02587d52445a1aaa86fa4a33b6879 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:15:49.583487 15057 raft_consensus.cc:729] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 3ce02587d52445a1aaa86fa4a33b6879 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce02587d52445a1aaa86fa4a33b6879, State: Initialized, Role: FOLLOWER
I20260806 21:15:49.583698 15056 raft_consensus.cc:348] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 31f2e75cea374d9ebbed7a9da90c2c23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31f2e75cea374d9ebbed7a9da90c2c23" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 34079 } } peers { permanent_uuid: "3ce02587d52445a1aaa86fa4a33b6879" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 44859 } } peers { permanent_uuid: "92e4e653051449fc88f176c965ad676f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 41801 } }
I20260806 21:15:49.583966 15056 raft_consensus.cc:374] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 31f2e75cea374d9ebbed7a9da90c2c23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:15:49.584036 15056 raft_consensus.cc:729] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 31f2e75cea374d9ebbed7a9da90c2c23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31f2e75cea374d9ebbed7a9da90c2c23, State: Initialized, Role: FOLLOWER
I20260806 21:15:49.584030 15057 consensus_queue.cc:260] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 3ce02587d52445a1aaa86fa4a33b6879 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31f2e75cea374d9ebbed7a9da90c2c23" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 34079 } } peers { permanent_uuid: "3ce02587d52445a1aaa86fa4a33b6879" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 44859 } } peers { permanent_uuid: "92e4e653051449fc88f176c965ad676f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 41801 } }
I20260806 21:15:49.584496 15056 consensus_queue.cc:260] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 31f2e75cea374d9ebbed7a9da90c2c23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31f2e75cea374d9ebbed7a9da90c2c23" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 34079 } } peers { permanent_uuid: "3ce02587d52445a1aaa86fa4a33b6879" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 44859 } } peers { permanent_uuid: "92e4e653051449fc88f176c965ad676f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 41801 } }
I20260806 21:15:49.585465 15057 ts_tablet_manager.cc:1428] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 3ce02587d52445a1aaa86fa4a33b6879: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 21:15:49.585537 15056 ts_tablet_manager.cc:1428] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 31f2e75cea374d9ebbed7a9da90c2c23: Time spent starting tablet: real 0.005s user 0.003s sys 0.002s
I20260806 21:15:49.585418 15058 raft_consensus.cc:348] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 92e4e653051449fc88f176c965ad676f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31f2e75cea374d9ebbed7a9da90c2c23" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 34079 } } peers { permanent_uuid: "3ce02587d52445a1aaa86fa4a33b6879" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 44859 } } peers { permanent_uuid: "92e4e653051449fc88f176c965ad676f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 41801 } }
I20260806 21:15:49.585785 15058 raft_consensus.cc:374] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 92e4e653051449fc88f176c965ad676f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:15:49.585846 15058 raft_consensus.cc:729] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 92e4e653051449fc88f176c965ad676f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92e4e653051449fc88f176c965ad676f, State: Initialized, Role: FOLLOWER
I20260806 21:15:49.585955 14821 heartbeater.cc:503] Master 127.16.178.62:46001 was elected leader, sending a full tablet report...
I20260806 21:15:49.585955 14616 heartbeater.cc:503] Master 127.16.178.62:46001 was elected leader, sending a full tablet report...
I20260806 21:15:49.586338 15058 consensus_queue.cc:260] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 92e4e653051449fc88f176c965ad676f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31f2e75cea374d9ebbed7a9da90c2c23" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 34079 } } peers { permanent_uuid: "3ce02587d52445a1aaa86fa4a33b6879" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 44859 } } peers { permanent_uuid: "92e4e653051449fc88f176c965ad676f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 41801 } }
I20260806 21:15:49.587538 15058 ts_tablet_manager.cc:1428] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 92e4e653051449fc88f176c965ad676f: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 21:15:49.588037 15035 heartbeater.cc:503] Master 127.16.178.62:46001 was elected leader, sending a full tablet report...
I20260806 21:15:49.659655 15068 raft_consensus.cc:482] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 92e4e653051449fc88f176c965ad676f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:15:49.659896 15068 raft_consensus.cc:504] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 92e4e653051449fc88f176c965ad676f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31f2e75cea374d9ebbed7a9da90c2c23" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 34079 } } peers { permanent_uuid: "3ce02587d52445a1aaa86fa4a33b6879" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 44859 } } peers { permanent_uuid: "92e4e653051449fc88f176c965ad676f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 41801 } }
I20260806 21:15:49.661319 15068 leader_election.cc:288] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 92e4e653051449fc88f176c965ad676f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31f2e75cea374d9ebbed7a9da90c2c23 (127.16.178.1:34079), 3ce02587d52445a1aaa86fa4a33b6879 (127.16.178.2:44859)
I20260806 21:15:49.665688 15038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:49.661565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.2:44859 (local address 127.16.178.3:47525)
0806 21:15:49.661775 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:49.661790 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:49.661887 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:49.662158 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:49.662162 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:49.662180 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:15:49.662305 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:49.662312 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:49.663201 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:49.663204 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:49.664115 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:49.664127 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:49.665207 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:49.665211 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:49.665319 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:49.665345 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 21:15:49.665418 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 21:15:49.665512 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121}
I20260806 21:15:49.665918 15055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:49.661817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:46853 (local address 127.16.178.1:34079)
0806 21:15:49.661954 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:49.661959 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:49.662462 (+ 503us) server_negotiation.cc:407] Connection header received
0806 21:15:49.662569 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:49.662572 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:49.662626 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:49.662701 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:15:49.662968 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:49.663760 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:49.664941 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:49.665375 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:49.665496 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:49.665551 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:49.665721 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 21:15:49.665802 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 21:15:49.665923 15070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:49.661572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:34079 (local address 127.16.178.3:46853)
0806 21:15:49.662371 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:49.662384 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:49.662484 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:49.662718 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:49.662720 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:49.662738 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:15:49.662857 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:49.662862 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:49.663905 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:49.663909 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:49.664794 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:49.664805 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:49.665502 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:49.665506 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:49.665610 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:49.665629 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 21:15:49.665694 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 21:15:49.665764 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":740,"mutex_wait_us":398,"thread_start_us":163,"threads_started":1}
I20260806 21:15:49.666529 15054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:49.661735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:47525 (local address 127.16.178.2:44859)
0806 21:15:49.661874 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:49.661879 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:49.661897 (+ 18us) server_negotiation.cc:407] Connection header received
0806 21:15:49.661990 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:49.661994 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:49.662047 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:49.662127 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:15:49.662455 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:49.663100 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:49.664242 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:49.664950 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:49.665083 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:49.665141 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:49.666319 (+ 1178us) server_negotiation.cc:299] Negotiation successful
0806 21:15:49.666408 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 21:15:49.666862 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ed5d4cab1e43ff9d69fa3e40b2e314" candidate_uuid: "92e4e653051449fc88f176c965ad676f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31f2e75cea374d9ebbed7a9da90c2c23" is_pre_election: true
I20260806 21:15:49.667107 14753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ed5d4cab1e43ff9d69fa3e40b2e314" candidate_uuid: "92e4e653051449fc88f176c965ad676f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce02587d52445a1aaa86fa4a33b6879" is_pre_election: true
I20260806 21:15:49.667315 14557 raft_consensus.cc:2454] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 31f2e75cea374d9ebbed7a9da90c2c23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92e4e653051449fc88f176c965ad676f in term 0.
I20260806 21:15:49.667440 14753 raft_consensus.cc:2454] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 3ce02587d52445a1aaa86fa4a33b6879 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92e4e653051449fc88f176c965ad676f in term 0.
I20260806 21:15:49.667932 14858 leader_election.cc:302] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 92e4e653051449fc88f176c965ad676f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31f2e75cea374d9ebbed7a9da90c2c23, 92e4e653051449fc88f176c965ad676f; no voters:
I20260806 21:15:49.668322 15068 raft_consensus.cc:2781] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 92e4e653051449fc88f176c965ad676f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:15:49.668416 15068 raft_consensus.cc:482] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 92e4e653051449fc88f176c965ad676f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:15:49.668488 15068 raft_consensus.cc:3037] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 92e4e653051449fc88f176c965ad676f [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:15:49.669694 15068 raft_consensus.cc:504] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 92e4e653051449fc88f176c965ad676f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31f2e75cea374d9ebbed7a9da90c2c23" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 34079 } } peers { permanent_uuid: "3ce02587d52445a1aaa86fa4a33b6879" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 44859 } } peers { permanent_uuid: "92e4e653051449fc88f176c965ad676f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 41801 } }
I20260806 21:15:49.670077 15068 leader_election.cc:288] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 92e4e653051449fc88f176c965ad676f [CANDIDATE]: Term 1 election: Requested vote from peers 31f2e75cea374d9ebbed7a9da90c2c23 (127.16.178.1:34079), 3ce02587d52445a1aaa86fa4a33b6879 (127.16.178.2:44859)
I20260806 21:15:49.670513 14753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ed5d4cab1e43ff9d69fa3e40b2e314" candidate_uuid: "92e4e653051449fc88f176c965ad676f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce02587d52445a1aaa86fa4a33b6879"
I20260806 21:15:49.670562 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ed5d4cab1e43ff9d69fa3e40b2e314" candidate_uuid: "92e4e653051449fc88f176c965ad676f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31f2e75cea374d9ebbed7a9da90c2c23"
I20260806 21:15:49.670648 14753 raft_consensus.cc:3037] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 3ce02587d52445a1aaa86fa4a33b6879 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:15:49.670707 14557 raft_consensus.cc:3037] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 31f2e75cea374d9ebbed7a9da90c2c23 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:15:49.671777 14753 raft_consensus.cc:2454] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 3ce02587d52445a1aaa86fa4a33b6879 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92e4e653051449fc88f176c965ad676f in term 1.
I20260806 21:15:49.672160 14860 leader_election.cc:302] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 92e4e653051449fc88f176c965ad676f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce02587d52445a1aaa86fa4a33b6879, 92e4e653051449fc88f176c965ad676f; no voters:
I20260806 21:15:49.672225 14557 raft_consensus.cc:2454] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 31f2e75cea374d9ebbed7a9da90c2c23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92e4e653051449fc88f176c965ad676f in term 1.
I20260806 21:15:49.672442 15068 raft_consensus.cc:2781] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 92e4e653051449fc88f176c965ad676f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:15:49.672683 15068 raft_consensus.cc:686] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 92e4e653051449fc88f176c965ad676f [term 1 LEADER]: Becoming Leader. State: Replica: 92e4e653051449fc88f176c965ad676f, State: Running, Role: LEADER
I20260806 21:15:49.673069 15068 consensus_queue.cc:237] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 92e4e653051449fc88f176c965ad676f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31f2e75cea374d9ebbed7a9da90c2c23" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 34079 } } peers { permanent_uuid: "3ce02587d52445a1aaa86fa4a33b6879" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 44859 } } peers { permanent_uuid: "92e4e653051449fc88f176c965ad676f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 41801 } }
I20260806 21:15:49.676362 14327 catalog_manager.cc:5495] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 92e4e653051449fc88f176c965ad676f reported cstate change: term changed from 0 to 1, leader changed from <none> to 92e4e653051449fc88f176c965ad676f (127.16.178.3). New cstate: current_term: 1 leader_uuid: "92e4e653051449fc88f176c965ad676f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31f2e75cea374d9ebbed7a9da90c2c23" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 34079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce02587d52445a1aaa86fa4a33b6879" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 44859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92e4e653051449fc88f176c965ad676f" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 41801 } health_report { overall_health: HEALTHY } } }
I20260806 21:15:49.696612 15053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:49.692527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42882 (local address 127.16.178.3:41801)
0806 21:15:49.692654 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:49.692657 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:49.692673 (+ 16us) server_negotiation.cc:407] Connection header received
0806 21:15:49.692810 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:49.692813 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:49.692873 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:49.692955 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:15:49.693236 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:49.693774 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:49.694914 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:49.695138 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:49.695219 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:49.695641 (+ 422us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:15:49.696203 (+ 562us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:15:49.696330 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:49.696384 (+ 54us) server_negotiation.cc:299] Negotiation successful
0806 21:15:49.696450 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 21:15:49.703203 14557 raft_consensus.cc:1261] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 31f2e75cea374d9ebbed7a9da90c2c23 [term 1 FOLLOWER]: Refusing update from remote peer 92e4e653051449fc88f176c965ad676f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:15:49.703229 14753 raft_consensus.cc:1261] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 3ce02587d52445a1aaa86fa4a33b6879 [term 1 FOLLOWER]: Refusing update from remote peer 92e4e653051449fc88f176c965ad676f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:15:49.704008 15068 consensus_queue.cc:1035] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 92e4e653051449fc88f176c965ad676f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce02587d52445a1aaa86fa4a33b6879" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 44859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:15:49.704267 15071 consensus_queue.cc:1035] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 92e4e653051449fc88f176c965ad676f [LEADER]: Connected to new peer: Peer: permanent_uuid: "31f2e75cea374d9ebbed7a9da90c2c23" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 34079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:15:49.715247 15074 mvcc.cc:205] Tried to move back new op lower bound from 7315664689975926784 to 7315664689863794688. Current Snapshot: MvccSnapshot[applied={T|T < 7315664689975926784}]
I20260806 21:15:49.715632 15075 mvcc.cc:205] Tried to move back new op lower bound from 7315664689975926784 to 7315664689863794688. Current Snapshot: MvccSnapshot[applied={T|T < 7315664689975926784 or (T in {7315664689975926784})}]
I20260806 21:15:49.718776 15079 mvcc.cc:205] Tried to move back new op lower bound from 7315664689975926784 to 7315664689863794688. Current Snapshot: MvccSnapshot[applied={T|T < 7315664689975926784}]
I20260806 21:15:49.728021 14326 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34270:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:15:49.728243 14326 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34270:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:15:49.731078 14326 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e0e45ebb880d4a2bb08cbac758fd60ea: Sending DeleteTablet for 3 replicas of tablet 97ed5d4cab1e43ff9d69fa3e40b2e314
I20260806 21:15:49.732147 14928 tablet_service.cc:1514] Processing DeleteTablet for tablet 97ed5d4cab1e43ff9d69fa3e40b2e314 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:15:49 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:38724
I20260806 21:15:49.732151 14523 tablet_service.cc:1514] Processing DeleteTablet for tablet 97ed5d4cab1e43ff9d69fa3e40b2e314 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:15:49 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:38936
I20260806 21:15:49.732152 14721 tablet_service.cc:1514] Processing DeleteTablet for tablet 97ed5d4cab1e43ff9d69fa3e40b2e314 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:15:49 UTC) from {username='kudu', principal='kudu/127.16.178.62@KRBTEST.COM'} at 127.0.0.1:39254
I20260806 21:15:49.732700 15091 tablet_replica.cc:330] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 3ce02587d52445a1aaa86fa4a33b6879: stopping tablet replica
I20260806 21:15:49.732900 15092 tablet_replica.cc:330] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 31f2e75cea374d9ebbed7a9da90c2c23: stopping tablet replica
I20260806 21:15:49.733155 15091 raft_consensus.cc:2229] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 3ce02587d52445a1aaa86fa4a33b6879 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:15:49.733311 15092 raft_consensus.cc:2229] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 31f2e75cea374d9ebbed7a9da90c2c23 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:15:49.733616 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 14423
I20260806 21:15:49.733738 15091 raft_consensus.cc:2258] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 3ce02587d52445a1aaa86fa4a33b6879 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:15:49.738049 15091 ts_tablet_manager.cc:1905] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 3ce02587d52445a1aaa86fa4a33b6879: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:15:49.741003 15091 ts_tablet_manager.cc:1918] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 3ce02587d52445a1aaa86fa4a33b6879: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:15:49.741398 15093 tablet_replica.cc:330] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 92e4e653051449fc88f176c965ad676f: stopping tablet replica
I20260806 21:15:49.741782 15093 raft_consensus.cc:2229] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 92e4e653051449fc88f176c965ad676f [term 1 LEADER]: Raft consensus shutting down.
I20260806 21:15:49.742265 15093 raft_consensus.cc:2258] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 92e4e653051449fc88f176c965ad676f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:15:49.742574 15091 log.cc:1196] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 3ce02587d52445a1aaa86fa4a33b6879: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-1/wal/wals/97ed5d4cab1e43ff9d69fa3e40b2e314
I20260806 21:15:49.742996 15091 ts_tablet_manager.cc:1939] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 3ce02587d52445a1aaa86fa4a33b6879: Deleting consensus metadata
I20260806 21:15:49.744071 15093 ts_tablet_manager.cc:1905] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 92e4e653051449fc88f176c965ad676f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:15:49.744560 14308 catalog_manager.cc:4841] TS 3ce02587d52445a1aaa86fa4a33b6879 (127.16.178.2:44859): tablet 97ed5d4cab1e43ff9d69fa3e40b2e314 (table test-table [id=d41cbf84b2e0478c88440c1c35f3ef84]) successfully deleted
I20260806 21:15:49.747198 15093 ts_tablet_manager.cc:1918] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 92e4e653051449fc88f176c965ad676f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:15:49.747325 15093 log.cc:1196] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 92e4e653051449fc88f176c965ad676f: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786050686797304-17096-0/minicluster-data/ts-2/wal/wals/97ed5d4cab1e43ff9d69fa3e40b2e314
I20260806 21:15:49.747720 15093 ts_tablet_manager.cc:1939] T 97ed5d4cab1e43ff9d69fa3e40b2e314 P 92e4e653051449fc88f176c965ad676f: Deleting consensus metadata
W20260806 21:15:49.748150 14304 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.178.1:34079 (error 108)
I20260806 21:15:49.749161 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 14623
I20260806 21:15:49.749182 14307 catalog_manager.cc:4841] TS 92e4e653051449fc88f176c965ad676f (127.16.178.3:41801): tablet 97ed5d4cab1e43ff9d69fa3e40b2e314 (table test-table [id=d41cbf84b2e0478c88440c1c35f3ef84]) successfully deleted
I20260806 21:15:49.749372 15051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:49.749012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:34079 (local address 127.0.0.1:43108)
0806 21:15:49.749201 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:49.749271 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.1:34079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
W20260806 21:15:49.756188 14304 catalog_manager.cc:4584] TS 31f2e75cea374d9ebbed7a9da90c2c23 (127.16.178.1:34079): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 97ed5d4cab1e43ff9d69fa3e40b2e314: Network error: Client connection negotiation failed: client connection to 127.16.178.1:34079: connect: Connection refused (error 111)
I20260806 21:15:49.790299 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 14830
I20260806 21:15:49.807989 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 14279
2026-08-06T21:15:49Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2587 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-06T21:15:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:15:49Z Disabled control of system clock
I20260806 21:15:49.852428 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:36403
--webserver_interface=127.16.178.62
--webserver_port=0
--builtin_ntp_servers=127.16.178.20:36789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:36403
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 21:15:49.968005 15101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:15:49.968350 15101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:15:49.968387 15101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:15:49.973326 15101 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:15:49.973428 15101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:15:49.973452 15101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:15:49.973479 15101 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:15:49.973515 15101 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:15:49.981159 15101 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:36789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:36403
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:36403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.178.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:15:49.982333 15101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:15:49.983765 15101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:15:49.992764 15112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:49.992726 15109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:49.992734 15116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:49.992985 15101 server_base.cc:1034] running on GCE node
I20260806 21:15:49.993603 15101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:15:49.994794 15101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:15:49.996007 15101 hybrid_clock.cc:648] HybridClock initialized: now 1786050949995974 us; error 41 us; skew 500 ppm
I20260806 21:15:49.997999 15101 webserver.cc:460] Webserver started at http://127.16.178.62:35323/ using document root <none> and password file <none>
I20260806 21:15:49.998624 15101 fs_manager.cc:359] Metadata directory not provided
I20260806 21:15:49.998698 15101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:15:49.998903 15101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:15:50.001199 15101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "8ddec048fb774a059a1a271e048c83b8"
format_stamp: "Formatted at 2026-08-06 21:15:49 on dist-test-slave-lgr5"
server_key: "c964beff5bdbecb7560ddc0e8bd493cf"
server_key_iv: "df77b45bd7f720f930b86e2ec167b027"
server_key_version: "encryptionkey@0"
I20260806 21:15:50.001784 15101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "8ddec048fb774a059a1a271e048c83b8"
format_stamp: "Formatted at 2026-08-06 21:15:49 on dist-test-slave-lgr5"
server_key: "c964beff5bdbecb7560ddc0e8bd493cf"
server_key_iv: "df77b45bd7f720f930b86e2ec167b027"
server_key_version: "encryptionkey@0"
I20260806 21:15:50.005756 15101 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:15:50.008491 15123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:50.009835 15101 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:15:50.009979 15101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "8ddec048fb774a059a1a271e048c83b8"
format_stamp: "Formatted at 2026-08-06 21:15:49 on dist-test-slave-lgr5"
server_key: "c964beff5bdbecb7560ddc0e8bd493cf"
server_key_iv: "df77b45bd7f720f930b86e2ec167b027"
server_key_version: "encryptionkey@0"
I20260806 21:15:50.010118 15101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:15:50.043109 15101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:15:50.044044 15101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:15:50.044299 15101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:15:50.052186 15101 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.62:36403
I20260806 21:15:50.052242 15213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.62:36403 every 8 connection(s)
I20260806 21:15:50.053284 15101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:15:50.056654 15214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:15:50.059355 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 15101
I20260806 21:15:50.059499 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:15:50.059836 17096 external_mini_cluster.cc:1428] Setting key e34e94d571f1c69d7c27f624a1feb9e5
I20260806 21:15:50.063443 15214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ddec048fb774a059a1a271e048c83b8: Bootstrap starting.
I20260806 21:15:50.066228 15214 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ddec048fb774a059a1a271e048c83b8: Neither blocks nor log segments found. Creating new log.
I20260806 21:15:50.067134 15214 log.cc:824] T 00000000000000000000000000000000 P 8ddec048fb774a059a1a271e048c83b8: Log is configured to *not* fsync() on all Append() calls
I20260806 21:15:50.069315 15214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ddec048fb774a059a1a271e048c83b8: No bootstrap required, opened a new log
I20260806 21:15:50.070406 15218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:50.061524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37866 (local address 127.16.178.62:36403)
0806 21:15:50.062290 (+ 766us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:50.062307 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:50.062350 (+ 43us) server_negotiation.cc:407] Connection header received
0806 21:15:50.063039 (+ 689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:50.063059 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:50.063508 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:50.063882 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:15:50.064421 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:50.065479 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:50.066838 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:50.067410 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:50.067579 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:50.068484 (+ 905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:15:50.068489 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:15:50.068506 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:15:50.068942 (+ 436us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:15:50.069004 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:15:50.069027 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:15:50.069121 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:50.069266 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 21:15:50.069580 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":145,"threads_started":1}
I20260806 21:15:50.073216 15214 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ddec048fb774a059a1a271e048c83b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ddec048fb774a059a1a271e048c83b8" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 36403 } }
I20260806 21:15:50.073540 15214 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ddec048fb774a059a1a271e048c83b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:15:50.073606 15214 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ddec048fb774a059a1a271e048c83b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ddec048fb774a059a1a271e048c83b8, State: Initialized, Role: FOLLOWER
I20260806 21:15:50.074115 15214 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ddec048fb774a059a1a271e048c83b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ddec048fb774a059a1a271e048c83b8" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 36403 } }
I20260806 21:15:50.074292 15214 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ddec048fb774a059a1a271e048c83b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:15:50.074381 15214 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ddec048fb774a059a1a271e048c83b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:15:50.074519 15214 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ddec048fb774a059a1a271e048c83b8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:15:50.075716 15214 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ddec048fb774a059a1a271e048c83b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ddec048fb774a059a1a271e048c83b8" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 36403 } }
I20260806 21:15:50.076205 15214 leader_election.cc:302] T 00000000000000000000000000000000 P 8ddec048fb774a059a1a271e048c83b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ddec048fb774a059a1a271e048c83b8; no voters:
I20260806 21:15:50.076666 15214 leader_election.cc:288] T 00000000000000000000000000000000 P 8ddec048fb774a059a1a271e048c83b8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:15:50.076824 15220 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ddec048fb774a059a1a271e048c83b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:15:50.077139 15220 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ddec048fb774a059a1a271e048c83b8 [term 1 LEADER]: Becoming Leader. State: Replica: 8ddec048fb774a059a1a271e048c83b8, State: Running, Role: LEADER
I20260806 21:15:50.077584 15220 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ddec048fb774a059a1a271e048c83b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ddec048fb774a059a1a271e048c83b8" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 36403 } }
I20260806 21:15:50.078305 15214 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ddec048fb774a059a1a271e048c83b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:15:50.079645 15221 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ddec048fb774a059a1a271e048c83b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ddec048fb774a059a1a271e048c83b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ddec048fb774a059a1a271e048c83b8" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 36403 } } }
I20260806 21:15:50.079869 15221 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ddec048fb774a059a1a271e048c83b8 [sys.catalog]: This master's current role is: LEADER
I20260806 21:15:50.079646 15224 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ddec048fb774a059a1a271e048c83b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ddec048fb774a059a1a271e048c83b8. Latest consensus state: current_term: 1 leader_uuid: "8ddec048fb774a059a1a271e048c83b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ddec048fb774a059a1a271e048c83b8" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 36403 } } }
I20260806 21:15:50.080279 15224 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ddec048fb774a059a1a271e048c83b8 [sys.catalog]: This master's current role is: LEADER
W20260806 21:15:50.083570 15241 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8ddec048fb774a059a1a271e048c83b8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 21:15:50.083658 15241 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 21:15:50.083732 15232 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:15:50.084601 15232 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:15:50.090797 15232 catalog_manager.cc:1347] Generated new cluster ID: b3f80eb44ed8410eae85f10a3c6105b4
I20260806 21:15:50.090888 15232 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:15:50.101753 15232 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:15:50.102720 15232 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:15:50.113982 15232 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ddec048fb774a059a1a271e048c83b8: Generated new TSK 0
I20260806 21:15:50.114724 15232 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:15:50.120458 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.1:0
--local_ip_for_outbound_sockets=127.16.178.1
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:36403
--builtin_ntp_servers=127.16.178.20:36789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 21:15:50.259737 15246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:15:50.260025 15246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:15:50.260062 15246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:15:50.264155 15246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:15:50.264225 15246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:15:50.264282 15246 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.1
I20260806 21:15:50.269176 15246 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:36789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.178.1
--webserver_port=0
--tserver_master_addrs=127.16.178.62:36403
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.1
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:15:50.270316 15246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:15:50.271692 15246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:15:50.282342 15254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:50.282366 15259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:50.282366 15256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:50.282596 15246 server_base.cc:1034] running on GCE node
I20260806 21:15:50.283048 15246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:15:50.283744 15246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:15:50.284914 15246 hybrid_clock.cc:648] HybridClock initialized: now 1786050950284877 us; error 48 us; skew 500 ppm
I20260806 21:15:50.286955 15246 webserver.cc:460] Webserver started at http://127.16.178.1:35733/ using document root <none> and password file <none>
I20260806 21:15:50.287595 15246 fs_manager.cc:359] Metadata directory not provided
I20260806 21:15:50.287670 15246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:15:50.287882 15246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:15:50.290220 15246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-0/data/instance:
uuid: "ba056c1a4b694e5995db1f06ff2d06af"
format_stamp: "Formatted at 2026-08-06 21:15:50 on dist-test-slave-lgr5"
server_key: "859bea17cb5c6e0d5a6d9713d1e39a41"
server_key_iv: "8f60956f028ee595f3b4b6fc24d5b9d6"
server_key_version: "encryptionkey@0"
I20260806 21:15:50.290860 15246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance:
uuid: "ba056c1a4b694e5995db1f06ff2d06af"
format_stamp: "Formatted at 2026-08-06 21:15:50 on dist-test-slave-lgr5"
server_key: "859bea17cb5c6e0d5a6d9713d1e39a41"
server_key_iv: "8f60956f028ee595f3b4b6fc24d5b9d6"
server_key_version: "encryptionkey@0"
I20260806 21:15:50.295002 15246 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 21:15:50.297638 15269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:50.298854 15246 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 21:15:50.298985 15246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-0/wal
uuid: "ba056c1a4b694e5995db1f06ff2d06af"
format_stamp: "Formatted at 2026-08-06 21:15:50 on dist-test-slave-lgr5"
server_key: "859bea17cb5c6e0d5a6d9713d1e39a41"
server_key_iv: "8f60956f028ee595f3b4b6fc24d5b9d6"
server_key_version: "encryptionkey@0"
I20260806 21:15:50.299093 15246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:15:50.333855 15246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:15:50.334650 15246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:15:50.334846 15246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:15:50.335505 15246 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:15:50.336542 15246 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:15:50.336592 15246 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:50.336647 15246 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:15:50.336675 15246 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:50.345276 15246 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.1:38751
I20260806 21:15:50.345314 15426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.1:38751 every 8 connection(s)
I20260806 21:15:50.346191 15246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-0/data/info.pb
I20260806 21:15:50.349472 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 15246
I20260806 21:15:50.349601 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-0/wal/instance
I20260806 21:15:50.349906 17096 external_mini_cluster.cc:1428] Setting key afb1c03de17644277047bd39fbc9b06b
I20260806 21:15:50.351979 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.2:0
--local_ip_for_outbound_sockets=127.16.178.2
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:36403
--builtin_ntp_servers=127.16.178.20:36789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 21:15:50.356194 15218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:50.348197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.1:47749 (local address 127.16.178.62:36403)
0806 21:15:50.348327 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:50.348331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:50.349399 (+ 1068us) server_negotiation.cc:407] Connection header received
0806 21:15:50.350952 (+ 1553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:50.350955 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:50.351021 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:50.351093 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:15:50.352074 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:50.352682 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:50.354016 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:50.354254 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:50.354362 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:50.355674 (+ 1312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:15:50.355678 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:15:50.355680 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:15:50.355696 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:15:50.355717 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:15:50.355725 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:15:50.355797 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:50.355964 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 21:15:50.356028 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 21:15:50.357544 15433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:50.348560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:36403 (local address 127.16.178.1:47749)
0806 21:15:50.349243 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:50.349286 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:50.350742 (+ 1456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:50.351289 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:50.351298 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:50.351403 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:15:50.351927 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:50.351941 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:50.352809 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:50.352812 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:50.353864 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:50.353879 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:50.354425 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:50.354428 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:50.354649 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:50.355335 (+ 686us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 21:15:50.355337 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:15:50.355393 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 21:15:50.355480 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:15:50.355808 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:15:50.355818 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:15:50.355824 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 21:15:50.355955 (+ 131us) client_negotiation.cc:240] Negotiation successful
0806 21:15:50.356334 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":140,"threads_started":1}
I20260806 21:15:50.359135 15429 heartbeater.cc:347] Connected to a master server at 127.16.178.62:36403
I20260806 21:15:50.359485 15429 heartbeater.cc:464] Registering TS with master...
I20260806 21:15:50.360373 15429 heartbeater.cc:511] Master 127.16.178.62:36403 requested a full tablet report, sending...
I20260806 21:15:50.362390 15161 ts_manager.cc:194] Registered new tserver with Master: ba056c1a4b694e5995db1f06ff2d06af (127.16.178.1:38751)
I20260806 21:15:50.363610 15161 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.178.1:47749
W20260806 21:15:50.472214 15434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:15:50.472512 15434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:15:50.472550 15434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:15:50.476743 15434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:15:50.476816 15434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:15:50.476876 15434 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.2
I20260806 21:15:50.482002 15434 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:36789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.178.2
--webserver_port=0
--tserver_master_addrs=127.16.178.62:36403
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.2
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:15:50.483160 15434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:15:50.484620 15434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:15:50.494412 15441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:50.494421 15445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:50.494439 15442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:50.494769 15434 server_base.cc:1034] running on GCE node
I20260806 21:15:50.495186 15434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:15:50.495849 15434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:15:50.496999 15434 hybrid_clock.cc:648] HybridClock initialized: now 1786050950496972 us; error 39 us; skew 500 ppm
I20260806 21:15:50.498778 15434 webserver.cc:460] Webserver started at http://127.16.178.2:38015/ using document root <none> and password file <none>
I20260806 21:15:50.499353 15434 fs_manager.cc:359] Metadata directory not provided
I20260806 21:15:50.499416 15434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:15:50.499611 15434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:15:50.501740 15434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-1/data/instance:
uuid: "c2357e255f58455391ae1d8df887f205"
format_stamp: "Formatted at 2026-08-06 21:15:50 on dist-test-slave-lgr5"
server_key: "2514972471cf6f4d8a2d49aa23bbd1ac"
server_key_iv: "8c7ae83d9cb4adae9a9ee5fce6f50f7b"
server_key_version: "encryptionkey@0"
I20260806 21:15:50.502331 15434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance:
uuid: "c2357e255f58455391ae1d8df887f205"
format_stamp: "Formatted at 2026-08-06 21:15:50 on dist-test-slave-lgr5"
server_key: "2514972471cf6f4d8a2d49aa23bbd1ac"
server_key_iv: "8c7ae83d9cb4adae9a9ee5fce6f50f7b"
server_key_version: "encryptionkey@0"
I20260806 21:15:50.506451 15434 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:15:50.508967 15454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:50.510203 15434 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 21:15:50.510310 15434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-1/wal
uuid: "c2357e255f58455391ae1d8df887f205"
format_stamp: "Formatted at 2026-08-06 21:15:50 on dist-test-slave-lgr5"
server_key: "2514972471cf6f4d8a2d49aa23bbd1ac"
server_key_iv: "8c7ae83d9cb4adae9a9ee5fce6f50f7b"
server_key_version: "encryptionkey@0"
I20260806 21:15:50.510426 15434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:15:50.528505 15434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:15:50.529260 15434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:15:50.529451 15434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:15:50.530081 15434 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:15:50.531070 15434 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:15:50.531118 15434 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:50.531164 15434 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:15:50.531193 15434 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:50.540138 15434 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.2:32945
I20260806 21:15:50.540179 15629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.2:32945 every 8 connection(s)
I20260806 21:15:50.541105 15434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-1/data/info.pb
I20260806 21:15:50.550659 15218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:50.543057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.2:58403 (local address 127.16.178.62:36403)
0806 21:15:50.543197 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:50.543200 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:50.544341 (+ 1141us) server_negotiation.cc:407] Connection header received
0806 21:15:50.545752 (+ 1411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:50.545755 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:50.545809 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:50.545888 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:15:50.546887 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:50.547438 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:50.548725 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:50.548958 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:50.549061 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:50.550144 (+ 1083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:15:50.550147 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:15:50.550149 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:15:50.550169 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:15:50.550190 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:15:50.550199 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:15:50.550263 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:50.550418 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 21:15:50.550488 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 21:15:50.550741 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 15434
I20260806 21:15:50.550870 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-1/wal/instance
I20260806 21:15:50.551214 17096 external_mini_cluster.cc:1428] Setting key 0f3ebd0e5be54567a00763800991fb86
I20260806 21:15:50.551553 15634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:50.543424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:36403 (local address 127.16.178.2:58403)
0806 21:15:50.544189 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:50.544233 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:50.545539 (+ 1306us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:50.546072 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:50.546079 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:50.546174 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:15:50.546724 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:50.546739 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:50.547558 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:50.547560 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:50.548573 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:50.548585 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:50.549102 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:50.549105 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:50.549255 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:50.549853 (+ 598us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 21:15:50.549855 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:15:50.549899 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 21:15:50.549972 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:15:50.550272 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:15:50.550280 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:15:50.550284 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 21:15:50.550402 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 21:15:50.550645 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":112,"threads_started":1}
I20260806 21:15:50.552671 15631 heartbeater.cc:347] Connected to a master server at 127.16.178.62:36403
I20260806 21:15:50.552959 15631 heartbeater.cc:464] Registering TS with master...
I20260806 21:15:50.553507 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.178.3:0
--local_ip_for_outbound_sockets=127.16.178.3
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:36403
--builtin_ntp_servers=127.16.178.20:36789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 21:15:50.553630 15631 heartbeater.cc:511] Master 127.16.178.62:36403 requested a full tablet report, sending...
I20260806 21:15:50.554760 15161 ts_manager.cc:194] Registered new tserver with Master: c2357e255f58455391ae1d8df887f205 (127.16.178.2:32945)
I20260806 21:15:50.555332 15161 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.178.2:58403
W20260806 21:15:50.673324 15635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:15:50.673633 15635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:15:50.673676 15635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:15:50.677845 15635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:15:50.677930 15635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:15:50.677994 15635 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.178.3
I20260806 21:15:50.683256 15635 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:36789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.178.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.178.3
--webserver_port=0
--tserver_master_addrs=127.16.178.62:36403
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.178.3
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:15:50.684572 15635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:15:50.686131 15635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:15:50.697109 15641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:50.697186 15644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:50.697141 15640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:50.697664 15635 server_base.cc:1034] running on GCE node
I20260806 21:15:50.698058 15635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:15:50.698678 15635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:15:50.699854 15635 hybrid_clock.cc:648] HybridClock initialized: now 1786050950699829 us; error 39 us; skew 500 ppm
I20260806 21:15:50.701880 15635 webserver.cc:460] Webserver started at http://127.16.178.3:36649/ using document root <none> and password file <none>
I20260806 21:15:50.702491 15635 fs_manager.cc:359] Metadata directory not provided
I20260806 21:15:50.702569 15635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:15:50.702752 15635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:15:50.704939 15635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-2/data/instance:
uuid: "7ee10eeddace4536ab3946483f356acb"
format_stamp: "Formatted at 2026-08-06 21:15:50 on dist-test-slave-lgr5"
server_key: "c8d5673abbd4cd4c5fc8d98b51730320"
server_key_iv: "1aaa9fb2b8fa932e61ecc6dabe2f38f2"
server_key_version: "encryptionkey@0"
I20260806 21:15:50.705490 15635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance:
uuid: "7ee10eeddace4536ab3946483f356acb"
format_stamp: "Formatted at 2026-08-06 21:15:50 on dist-test-slave-lgr5"
server_key: "c8d5673abbd4cd4c5fc8d98b51730320"
server_key_iv: "1aaa9fb2b8fa932e61ecc6dabe2f38f2"
server_key_version: "encryptionkey@0"
I20260806 21:15:50.709308 15635 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:15:50.711691 15657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:50.713042 15635 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 21:15:50.713158 15635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-2/wal
uuid: "7ee10eeddace4536ab3946483f356acb"
format_stamp: "Formatted at 2026-08-06 21:15:50 on dist-test-slave-lgr5"
server_key: "c8d5673abbd4cd4c5fc8d98b51730320"
server_key_iv: "1aaa9fb2b8fa932e61ecc6dabe2f38f2"
server_key_version: "encryptionkey@0"
I20260806 21:15:50.713264 15635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:15:50.728192 15635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:15:50.728951 15635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:15:50.729147 15635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:15:50.729765 15635 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 21:15:50.730760 15635 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 21:15:50.730810 15635 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:50.730863 15635 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 21:15:50.730893 15635 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:50.739889 15635 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.3:34927
I20260806 21:15:50.739913 15845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.3:34927 every 8 connection(s)
I20260806 21:15:50.740906 15635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-2/data/info.pb
I20260806 21:15:50.741242 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 15635
I20260806 21:15:50.741341 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-2/wal/instance
I20260806 21:15:50.741680 17096 external_mini_cluster.cc:1428] Setting key e2ff4d1091fee76675e2f3a17b59290a
I20260806 21:15:50.750000 15218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:50.742795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:51049 (local address 127.16.178.62:36403)
0806 21:15:50.742929 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:50.742934 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:50.744182 (+ 1248us) server_negotiation.cc:407] Connection header received
0806 21:15:50.745555 (+ 1373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:50.745558 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:50.745606 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:50.745684 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:15:50.746528 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:50.747060 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:50.748205 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:50.748436 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:50.748525 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:50.749525 (+ 1000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:15:50.749528 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:15:50.749530 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:15:50.749543 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:15:50.749562 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:15:50.749569 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:15:50.749626 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:50.749798 (+ 172us) server_negotiation.cc:299] Negotiation successful
0806 21:15:50.749861 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 21:15:50.750775 15849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:50.743251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:36403 (local address 127.16.178.3:51049)
0806 21:15:50.743999 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:50.744051 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:50.745346 (+ 1295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:50.745829 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:50.745835 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:50.745916 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:15:50.746408 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:50.746421 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:50.747152 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:50.747154 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:50.748028 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:50.748037 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:50.748566 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:50.748569 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:50.748712 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:50.749264 (+ 552us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 21:15:50.749265 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:15:50.749305 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 21:15:50.749372 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:15:50.749633 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:15:50.749638 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:15:50.749641 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 21:15:50.749728 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 21:15:50.749944 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":128,"threads_started":1}
I20260806 21:15:50.751972 15846 heartbeater.cc:347] Connected to a master server at 127.16.178.62:36403
I20260806 21:15:50.752233 15846 heartbeater.cc:464] Registering TS with master...
I20260806 21:15:50.752761 15846 heartbeater.cc:511] Master 127.16.178.62:36403 requested a full tablet report, sending...
I20260806 21:15:50.753798 15161 ts_manager.cc:194] Registered new tserver with Master: 7ee10eeddace4536ab3946483f356acb (127.16.178.3:34927)
I20260806 21:15:50.754446 15161 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.178.3:51049
I20260806 21:15:50.755820 17096 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 21:15:50.760950 15218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:50.757684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37872 (local address 127.16.178.62:36403)
0806 21:15:50.757810 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:50.757814 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:50.757980 (+ 166us) server_negotiation.cc:407] Connection header received
0806 21:15:50.758092 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:50.758094 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:50.758134 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:50.758189 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:15:50.758398 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:50.758895 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:50.759640 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:50.759838 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:50.759902 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:50.760610 (+ 708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:15:50.760613 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:15:50.760615 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:15:50.760627 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:15:50.760645 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:15:50.760652 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:15:50.760707 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:50.760767 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 21:15:50.760815 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 21:15:50.763769 15160 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37872:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 21:15:50.766794 15160 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 21:15:50.781733 15866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:50.776274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41004 (local address 127.16.178.3:34927)
0806 21:15:50.776909 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:50.776921 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:50.776944 (+ 23us) server_negotiation.cc:407] Connection header received
0806 21:15:50.777019 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:50.777023 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:50.777186 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:50.777331 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:15:50.777549 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:50.778623 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:50.780202 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:50.780993 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:50.781096 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:50.781339 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:50.781439 (+ 100us) server_negotiation.cc:299] Negotiation successful
0806 21:15:50.781548 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":105,"threads_started":1}
I20260806 21:15:50.781730 15865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:50.775826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.3:34927 (local address 127.0.0.1:41004)
0806 21:15:50.776354 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:50.776370 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:50.776463 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:50.777312 (+ 849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:50.777316 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:50.777334 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:15:50.777430 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:50.777436 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:50.778748 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:50.778752 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:50.780069 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:50.780105 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:50.781112 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:50.781116 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:50.781234 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:50.781277 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 21:15:50.781408 (+ 131us) client_negotiation.cc:240] Negotiation successful
0806 21:15:50.781548 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"mutex_wait_us":75,"thread_start_us":115,"threads_started":1}
I20260806 21:15:50.781734 15861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:50.775413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:38751 (local address 127.0.0.1:37932)
0806 21:15:50.775890 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:50.775940 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:50.776066 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:50.776921 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:50.776928 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:50.776951 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:15:50.777119 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:50.777128 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:50.778465 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:50.778468 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:50.780069 (+ 1601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:50.780099 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:50.781175 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:50.781180 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:50.781284 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:50.781313 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 21:15:50.781370 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 21:15:50.781563 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":116,"thread_start_us":99,"threads_started":1}
I20260806 21:15:50.782335 15868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:50.776672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.2:32945 (local address 127.0.0.1:33624)
0806 21:15:50.776959 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:50.776971 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:50.777064 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:50.777700 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:50.777703 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:50.777720 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:15:50.777796 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:50.777801 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:50.779057 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:50.779060 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:50.780393 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:50.780405 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:50.781664 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:50.781667 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:50.781752 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:50.781777 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 21:15:50.782121 (+ 344us) client_negotiation.cc:240] Negotiation successful
0806 21:15:50.782209 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":84,"threads_started":1}
I20260806 21:15:50.782717 15864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:50.775561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37932 (local address 127.16.178.1:38751)
0806 21:15:50.776396 (+ 835us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:50.776408 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:50.776433 (+ 25us) server_negotiation.cc:407] Connection header received
0806 21:15:50.776527 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:50.776533 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:50.776745 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:50.777266 (+ 521us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:15:50.777405 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:50.778361 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:50.780249 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:50.781026 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:50.782080 (+ 1054us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:50.782284 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:50.782337 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 21:15:50.782555 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":726,"thread_start_us":148,"threads_started":1}
I20260806 21:15:50.782795 15867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:50.776336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33624 (local address 127.16.178.2:32945)
0806 21:15:50.776719 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:50.776735 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:50.777151 (+ 416us) server_negotiation.cc:407] Connection header received
0806 21:15:50.777231 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:50.777235 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:50.777399 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:50.777548 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:15:50.777937 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:50.778940 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:50.780540 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:50.781295 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:50.781399 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:50.781599 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:50.782536 (+ 937us) server_negotiation.cc:299] Negotiation successful
0806 21:15:50.782640 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":122,"threads_started":1}
I20260806 21:15:50.785097 15733 tablet_service.cc:1467] Processing CreateTablet for tablet 61802bd1abff435c807ce780327af458 (DEFAULT_TABLE table=test-table [id=c23f5b30dec64ac6b48fd789144b32e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:15:50.785344 15343 tablet_service.cc:1467] Processing CreateTablet for tablet 61802bd1abff435c807ce780327af458 (DEFAULT_TABLE table=test-table [id=c23f5b30dec64ac6b48fd789144b32e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:15:50.786262 15343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61802bd1abff435c807ce780327af458. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:15:50.786284 15733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61802bd1abff435c807ce780327af458. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:15:50.785912 15528 tablet_service.cc:1467] Processing CreateTablet for tablet 61802bd1abff435c807ce780327af458 (DEFAULT_TABLE table=test-table [id=c23f5b30dec64ac6b48fd789144b32e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 21:15:50.786862 15528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61802bd1abff435c807ce780327af458. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:15:50.792553 15870 tablet_bootstrap.cc:492] T 61802bd1abff435c807ce780327af458 P ba056c1a4b694e5995db1f06ff2d06af: Bootstrap starting.
I20260806 21:15:50.793727 15872 tablet_bootstrap.cc:492] T 61802bd1abff435c807ce780327af458 P 7ee10eeddace4536ab3946483f356acb: Bootstrap starting.
I20260806 21:15:50.795679 15872 tablet_bootstrap.cc:654] T 61802bd1abff435c807ce780327af458 P 7ee10eeddace4536ab3946483f356acb: Neither blocks nor log segments found. Creating new log.
I20260806 21:15:50.795773 15870 tablet_bootstrap.cc:654] T 61802bd1abff435c807ce780327af458 P ba056c1a4b694e5995db1f06ff2d06af: Neither blocks nor log segments found. Creating new log.
I20260806 21:15:50.796451 15873 tablet_bootstrap.cc:492] T 61802bd1abff435c807ce780327af458 P c2357e255f58455391ae1d8df887f205: Bootstrap starting.
I20260806 21:15:50.796528 15872 log.cc:824] T 61802bd1abff435c807ce780327af458 P 7ee10eeddace4536ab3946483f356acb: Log is configured to *not* fsync() on all Append() calls
I20260806 21:15:50.796612 15870 log.cc:824] T 61802bd1abff435c807ce780327af458 P ba056c1a4b694e5995db1f06ff2d06af: Log is configured to *not* fsync() on all Append() calls
I20260806 21:15:50.798911 15872 tablet_bootstrap.cc:492] T 61802bd1abff435c807ce780327af458 P 7ee10eeddace4536ab3946483f356acb: No bootstrap required, opened a new log
I20260806 21:15:50.799057 15873 tablet_bootstrap.cc:654] T 61802bd1abff435c807ce780327af458 P c2357e255f58455391ae1d8df887f205: Neither blocks nor log segments found. Creating new log.
I20260806 21:15:50.799192 15872 ts_tablet_manager.cc:1397] T 61802bd1abff435c807ce780327af458 P 7ee10eeddace4536ab3946483f356acb: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 21:15:50.799974 15873 log.cc:824] T 61802bd1abff435c807ce780327af458 P c2357e255f58455391ae1d8df887f205: Log is configured to *not* fsync() on all Append() calls
I20260806 21:15:50.800695 15870 tablet_bootstrap.cc:492] T 61802bd1abff435c807ce780327af458 P ba056c1a4b694e5995db1f06ff2d06af: No bootstrap required, opened a new log
I20260806 21:15:50.800906 15870 ts_tablet_manager.cc:1397] T 61802bd1abff435c807ce780327af458 P ba056c1a4b694e5995db1f06ff2d06af: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260806 21:15:50.802968 15872 raft_consensus.cc:348] T 61802bd1abff435c807ce780327af458 P 7ee10eeddace4536ab3946483f356acb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba056c1a4b694e5995db1f06ff2d06af" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 38751 } } peers { permanent_uuid: "c2357e255f58455391ae1d8df887f205" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 32945 } } peers { permanent_uuid: "7ee10eeddace4536ab3946483f356acb" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 34927 } }
I20260806 21:15:50.803300 15872 raft_consensus.cc:374] T 61802bd1abff435c807ce780327af458 P 7ee10eeddace4536ab3946483f356acb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:15:50.803362 15872 raft_consensus.cc:729] T 61802bd1abff435c807ce780327af458 P 7ee10eeddace4536ab3946483f356acb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ee10eeddace4536ab3946483f356acb, State: Initialized, Role: FOLLOWER
I20260806 21:15:50.803828 15872 consensus_queue.cc:260] T 61802bd1abff435c807ce780327af458 P 7ee10eeddace4536ab3946483f356acb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba056c1a4b694e5995db1f06ff2d06af" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 38751 } } peers { permanent_uuid: "c2357e255f58455391ae1d8df887f205" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 32945 } } peers { permanent_uuid: "7ee10eeddace4536ab3946483f356acb" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 34927 } }
I20260806 21:15:50.804553 15846 heartbeater.cc:503] Master 127.16.178.62:36403 was elected leader, sending a full tablet report...
I20260806 21:15:50.804538 15870 raft_consensus.cc:348] T 61802bd1abff435c807ce780327af458 P ba056c1a4b694e5995db1f06ff2d06af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba056c1a4b694e5995db1f06ff2d06af" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 38751 } } peers { permanent_uuid: "c2357e255f58455391ae1d8df887f205" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 32945 } } peers { permanent_uuid: "7ee10eeddace4536ab3946483f356acb" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 34927 } }
I20260806 21:15:50.804795 15870 raft_consensus.cc:374] T 61802bd1abff435c807ce780327af458 P ba056c1a4b694e5995db1f06ff2d06af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:15:50.804867 15870 raft_consensus.cc:729] T 61802bd1abff435c807ce780327af458 P ba056c1a4b694e5995db1f06ff2d06af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba056c1a4b694e5995db1f06ff2d06af, State: Initialized, Role: FOLLOWER
I20260806 21:15:50.804925 15872 ts_tablet_manager.cc:1428] T 61802bd1abff435c807ce780327af458 P 7ee10eeddace4536ab3946483f356acb: Time spent starting tablet: real 0.006s user 0.004s sys 0.001s
I20260806 21:15:50.804946 15873 tablet_bootstrap.cc:492] T 61802bd1abff435c807ce780327af458 P c2357e255f58455391ae1d8df887f205: No bootstrap required, opened a new log
I20260806 21:15:50.805142 15873 ts_tablet_manager.cc:1397] T 61802bd1abff435c807ce780327af458 P c2357e255f58455391ae1d8df887f205: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260806 21:15:50.805320 15870 consensus_queue.cc:260] T 61802bd1abff435c807ce780327af458 P ba056c1a4b694e5995db1f06ff2d06af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba056c1a4b694e5995db1f06ff2d06af" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 38751 } } peers { permanent_uuid: "c2357e255f58455391ae1d8df887f205" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 32945 } } peers { permanent_uuid: "7ee10eeddace4536ab3946483f356acb" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 34927 } }
I20260806 21:15:50.805995 15429 heartbeater.cc:503] Master 127.16.178.62:36403 was elected leader, sending a full tablet report...
I20260806 21:15:50.808564 15870 ts_tablet_manager.cc:1428] T 61802bd1abff435c807ce780327af458 P ba056c1a4b694e5995db1f06ff2d06af: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260806 21:15:50.809365 15873 raft_consensus.cc:348] T 61802bd1abff435c807ce780327af458 P c2357e255f58455391ae1d8df887f205 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba056c1a4b694e5995db1f06ff2d06af" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 38751 } } peers { permanent_uuid: "c2357e255f58455391ae1d8df887f205" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 32945 } } peers { permanent_uuid: "7ee10eeddace4536ab3946483f356acb" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 34927 } }
I20260806 21:15:50.809593 15873 raft_consensus.cc:374] T 61802bd1abff435c807ce780327af458 P c2357e255f58455391ae1d8df887f205 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:15:50.809636 15873 raft_consensus.cc:729] T 61802bd1abff435c807ce780327af458 P c2357e255f58455391ae1d8df887f205 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2357e255f58455391ae1d8df887f205, State: Initialized, Role: FOLLOWER
I20260806 21:15:50.810032 15873 consensus_queue.cc:260] T 61802bd1abff435c807ce780327af458 P c2357e255f58455391ae1d8df887f205 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba056c1a4b694e5995db1f06ff2d06af" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 38751 } } peers { permanent_uuid: "c2357e255f58455391ae1d8df887f205" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 32945 } } peers { permanent_uuid: "7ee10eeddace4536ab3946483f356acb" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 34927 } }
I20260806 21:15:50.810673 15631 heartbeater.cc:503] Master 127.16.178.62:36403 was elected leader, sending a full tablet report...
I20260806 21:15:50.810930 15873 ts_tablet_manager.cc:1428] T 61802bd1abff435c807ce780327af458 P c2357e255f58455391ae1d8df887f205: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260806 21:15:50.847522 15432 tablet.cc:2369] T 61802bd1abff435c807ce780327af458 P ba056c1a4b694e5995db1f06ff2d06af: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 21:15:50.926378 15879 raft_consensus.cc:482] T 61802bd1abff435c807ce780327af458 P 7ee10eeddace4536ab3946483f356acb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:15:50.926638 15879 raft_consensus.cc:504] T 61802bd1abff435c807ce780327af458 P 7ee10eeddace4536ab3946483f356acb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba056c1a4b694e5995db1f06ff2d06af" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 38751 } } peers { permanent_uuid: "c2357e255f58455391ae1d8df887f205" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 32945 } } peers { permanent_uuid: "7ee10eeddace4536ab3946483f356acb" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 34927 } }
I20260806 21:15:50.927949 15879 leader_election.cc:288] T 61802bd1abff435c807ce780327af458 P 7ee10eeddace4536ab3946483f356acb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba056c1a4b694e5995db1f06ff2d06af (127.16.178.1:38751), c2357e255f58455391ae1d8df887f205 (127.16.178.2:32945)
I20260806 21:15:50.931946 15849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:50.928259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:38751 (local address 127.16.178.3:46977)
0806 21:15:50.928400 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:50.928418 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:50.928539 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:50.928940 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:50.928944 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:50.928967 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:15:50.929103 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:50.929112 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:50.929971 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:50.929974 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:50.930802 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:50.930814 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:50.931504 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:50.931508 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:50.931617 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:50.931637 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 21:15:50.931705 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 21:15:50.931788 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 21:15:50.931946 15864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:50.928395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:46977 (local address 127.16.178.1:38751)
0806 21:15:50.928568 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:50.928574 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:50.928594 (+ 20us) server_negotiation.cc:407] Connection header received
0806 21:15:50.928660 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:50.928664 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:50.928722 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:50.928796 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:15:50.929292 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:50.929841 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:50.930970 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:50.931380 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:50.931500 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:50.931560 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:50.931737 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 21:15:50.931812 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 21:15:50.932161 15867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:50.928827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.178.3:35049 (local address 127.16.178.2:32945)
0806 21:15:50.928947 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:50.928951 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:50.929174 (+ 223us) server_negotiation.cc:407] Connection header received
0806 21:15:50.929291 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:50.929294 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:50.929371 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:50.929437 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 21:15:50.929725 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:50.930222 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:50.931232 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:50.931620 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:50.931727 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:50.931778 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:50.931953 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 21:15:50.932032 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 21:15:50.932168 15884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:50.928550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.2:32945 (local address 127.16.178.3:35049)
0806 21:15:50.929077 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:50.929089 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 21:15:50.929184 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:15:50.929471 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:15:50.929474 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:15:50.929498 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 21:15:50.929615 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:50.929620 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:50.930334 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:50.930338 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:50.931105 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:15:50.931114 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:15:50.931745 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:15:50.931749 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:15:50.931842 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:50.931860 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 21:15:50.931927 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 21:15:50.931997 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":158,"threads_started":1}
I20260806 21:15:50.932924 15369 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61802bd1abff435c807ce780327af458" candidate_uuid: "7ee10eeddace4536ab3946483f356acb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba056c1a4b694e5995db1f06ff2d06af" is_pre_election: true
I20260806 21:15:50.932930 15555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61802bd1abff435c807ce780327af458" candidate_uuid: "7ee10eeddace4536ab3946483f356acb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2357e255f58455391ae1d8df887f205" is_pre_election: true
I20260806 21:15:50.933284 15555 raft_consensus.cc:2454] T 61802bd1abff435c807ce780327af458 P c2357e255f58455391ae1d8df887f205 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ee10eeddace4536ab3946483f356acb in term 0.
I20260806 21:15:50.933284 15369 raft_consensus.cc:2454] T 61802bd1abff435c807ce780327af458 P ba056c1a4b694e5995db1f06ff2d06af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ee10eeddace4536ab3946483f356acb in term 0.
I20260806 21:15:50.933790 15663 leader_election.cc:302] T 61802bd1abff435c807ce780327af458 P 7ee10eeddace4536ab3946483f356acb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ee10eeddace4536ab3946483f356acb, ba056c1a4b694e5995db1f06ff2d06af; no voters:
I20260806 21:15:50.934062 15879 raft_consensus.cc:2781] T 61802bd1abff435c807ce780327af458 P 7ee10eeddace4536ab3946483f356acb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 21:15:50.934154 15879 raft_consensus.cc:482] T 61802bd1abff435c807ce780327af458 P 7ee10eeddace4536ab3946483f356acb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 21:15:50.934206 15879 raft_consensus.cc:3037] T 61802bd1abff435c807ce780327af458 P 7ee10eeddace4536ab3946483f356acb [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:15:50.935254 15879 raft_consensus.cc:504] T 61802bd1abff435c807ce780327af458 P 7ee10eeddace4536ab3946483f356acb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba056c1a4b694e5995db1f06ff2d06af" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 38751 } } peers { permanent_uuid: "c2357e255f58455391ae1d8df887f205" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 32945 } } peers { permanent_uuid: "7ee10eeddace4536ab3946483f356acb" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 34927 } }
I20260806 21:15:50.935599 15879 leader_election.cc:288] T 61802bd1abff435c807ce780327af458 P 7ee10eeddace4536ab3946483f356acb [CANDIDATE]: Term 1 election: Requested vote from peers ba056c1a4b694e5995db1f06ff2d06af (127.16.178.1:38751), c2357e255f58455391ae1d8df887f205 (127.16.178.2:32945)
I20260806 21:15:50.936034 15369 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61802bd1abff435c807ce780327af458" candidate_uuid: "7ee10eeddace4536ab3946483f356acb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba056c1a4b694e5995db1f06ff2d06af"
I20260806 21:15:50.936034 15555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61802bd1abff435c807ce780327af458" candidate_uuid: "7ee10eeddace4536ab3946483f356acb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2357e255f58455391ae1d8df887f205"
I20260806 21:15:50.936195 15555 raft_consensus.cc:3037] T 61802bd1abff435c807ce780327af458 P c2357e255f58455391ae1d8df887f205 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:15:50.936194 15369 raft_consensus.cc:3037] T 61802bd1abff435c807ce780327af458 P ba056c1a4b694e5995db1f06ff2d06af [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:15:50.937516 15555 raft_consensus.cc:2454] T 61802bd1abff435c807ce780327af458 P c2357e255f58455391ae1d8df887f205 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ee10eeddace4536ab3946483f356acb in term 1.
I20260806 21:15:50.937515 15369 raft_consensus.cc:2454] T 61802bd1abff435c807ce780327af458 P ba056c1a4b694e5995db1f06ff2d06af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ee10eeddace4536ab3946483f356acb in term 1.
I20260806 21:15:50.937865 15663 leader_election.cc:302] T 61802bd1abff435c807ce780327af458 P 7ee10eeddace4536ab3946483f356acb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ee10eeddace4536ab3946483f356acb, c2357e255f58455391ae1d8df887f205; no voters:
I20260806 21:15:50.938079 15879 raft_consensus.cc:2781] T 61802bd1abff435c807ce780327af458 P 7ee10eeddace4536ab3946483f356acb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:15:50.938324 15879 raft_consensus.cc:686] T 61802bd1abff435c807ce780327af458 P 7ee10eeddace4536ab3946483f356acb [term 1 LEADER]: Becoming Leader. State: Replica: 7ee10eeddace4536ab3946483f356acb, State: Running, Role: LEADER
I20260806 21:15:50.938710 15879 consensus_queue.cc:237] T 61802bd1abff435c807ce780327af458 P 7ee10eeddace4536ab3946483f356acb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba056c1a4b694e5995db1f06ff2d06af" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 38751 } } peers { permanent_uuid: "c2357e255f58455391ae1d8df887f205" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 32945 } } peers { permanent_uuid: "7ee10eeddace4536ab3946483f356acb" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 34927 } }
I20260806 21:15:50.942291 15159 catalog_manager.cc:5495] T 61802bd1abff435c807ce780327af458 P 7ee10eeddace4536ab3946483f356acb reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ee10eeddace4536ab3946483f356acb (127.16.178.3). New cstate: current_term: 1 leader_uuid: "7ee10eeddace4536ab3946483f356acb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba056c1a4b694e5995db1f06ff2d06af" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 38751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2357e255f58455391ae1d8df887f205" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 32945 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ee10eeddace4536ab3946483f356acb" member_type: VOTER last_known_addr { host: "127.16.178.3" port: 34927 } health_report { overall_health: HEALTHY } } }
I20260806 21:15:50.975626 15866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:50.971641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41014 (local address 127.16.178.3:34927)
0806 21:15:50.971773 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:50.971779 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:50.971794 (+ 15us) server_negotiation.cc:407] Connection header received
0806 21:15:50.971963 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:50.971966 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:50.972027 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:50.972168 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:15:50.972477 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:50.973069 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:50.973978 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:50.974201 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:50.974295 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:50.974658 (+ 363us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:15:50.975161 (+ 503us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:15:50.975278 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:50.975373 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 21:15:50.975436 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 21:15:50.982219 15555 raft_consensus.cc:1261] T 61802bd1abff435c807ce780327af458 P c2357e255f58455391ae1d8df887f205 [term 1 FOLLOWER]: Refusing update from remote peer 7ee10eeddace4536ab3946483f356acb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:15:50.982219 15369 raft_consensus.cc:1261] T 61802bd1abff435c807ce780327af458 P ba056c1a4b694e5995db1f06ff2d06af [term 1 FOLLOWER]: Refusing update from remote peer 7ee10eeddace4536ab3946483f356acb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:15:50.982991 15885 consensus_queue.cc:1035] T 61802bd1abff435c807ce780327af458 P 7ee10eeddace4536ab3946483f356acb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba056c1a4b694e5995db1f06ff2d06af" member_type: VOTER last_known_addr { host: "127.16.178.1" port: 38751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:15:50.983208 15879 consensus_queue.cc:1035] T 61802bd1abff435c807ce780327af458 P 7ee10eeddace4536ab3946483f356acb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2357e255f58455391ae1d8df887f205" member_type: VOTER last_known_addr { host: "127.16.178.2" port: 32945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:15:50.992009 15898 mvcc.cc:205] Tried to move back new op lower bound from 7315664695215407104 to 7315664695047938048. Current Snapshot: MvccSnapshot[applied={T|T < 7315664695215407104}]
I20260806 21:15:50.992367 15895 mvcc.cc:205] Tried to move back new op lower bound from 7315664695215407104 to 7315664695047938048. Current Snapshot: MvccSnapshot[applied={T|T < 7315664695215407104}]
I20260806 21:15:51.002272 15161 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37872:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:15:51.002530 15161 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37872:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 21:15:51.005445 15161 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8ddec048fb774a059a1a271e048c83b8: Sending DeleteTablet for 3 replicas of tablet 61802bd1abff435c807ce780327af458
I20260806 21:15:51.006520 15528 tablet_service.cc:1514] Processing DeleteTablet for tablet 61802bd1abff435c807ce780327af458 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:15:50 UTC) from {username='slave'} at 127.0.0.1:33624
I20260806 21:15:51.006520 15733 tablet_service.cc:1514] Processing DeleteTablet for tablet 61802bd1abff435c807ce780327af458 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:15:50 UTC) from {username='slave'} at 127.0.0.1:41004
I20260806 21:15:51.007145 15912 tablet_replica.cc:330] T 61802bd1abff435c807ce780327af458 P 7ee10eeddace4536ab3946483f356acb: stopping tablet replica
I20260806 21:15:51.007323 15343 tablet_service.cc:1514] Processing DeleteTablet for tablet 61802bd1abff435c807ce780327af458 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 21:15:50 UTC) from {username='slave'} at 127.0.0.1:37932
I20260806 21:15:51.007419 15910 tablet_replica.cc:330] T 61802bd1abff435c807ce780327af458 P c2357e255f58455391ae1d8df887f205: stopping tablet replica
I20260806 21:15:51.007580 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 15246
I20260806 21:15:51.007942 15910 raft_consensus.cc:2229] T 61802bd1abff435c807ce780327af458 P c2357e255f58455391ae1d8df887f205 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 21:15:51.008323 15912 raft_consensus.cc:2229] T 61802bd1abff435c807ce780327af458 P 7ee10eeddace4536ab3946483f356acb [term 1 LEADER]: Raft consensus shutting down.
I20260806 21:15:51.008750 15910 raft_consensus.cc:2258] T 61802bd1abff435c807ce780327af458 P c2357e255f58455391ae1d8df887f205 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:15:51.009164 15912 raft_consensus.cc:2258] T 61802bd1abff435c807ce780327af458 P 7ee10eeddace4536ab3946483f356acb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 21:15:51.012377 15910 ts_tablet_manager.cc:1905] T 61802bd1abff435c807ce780327af458 P c2357e255f58455391ae1d8df887f205: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:15:51.013466 15912 ts_tablet_manager.cc:1905] T 61802bd1abff435c807ce780327af458 P 7ee10eeddace4536ab3946483f356acb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 21:15:51.015918 15910 ts_tablet_manager.cc:1918] T 61802bd1abff435c807ce780327af458 P c2357e255f58455391ae1d8df887f205: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:15:51.016181 15910 log.cc:1196] T 61802bd1abff435c807ce780327af458 P c2357e255f58455391ae1d8df887f205: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-1/wal/wals/61802bd1abff435c807ce780327af458
I20260806 21:15:51.016531 15912 ts_tablet_manager.cc:1918] T 61802bd1abff435c807ce780327af458 P 7ee10eeddace4536ab3946483f356acb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 21:15:51.016638 15912 log.cc:1196] T 61802bd1abff435c807ce780327af458 P 7ee10eeddace4536ab3946483f356acb: Deleting WAL directory at /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786050686797304-17096-0/minicluster-data/ts-2/wal/wals/61802bd1abff435c807ce780327af458
I20260806 21:15:51.016804 15910 ts_tablet_manager.cc:1939] T 61802bd1abff435c807ce780327af458 P c2357e255f58455391ae1d8df887f205: Deleting consensus metadata
I20260806 21:15:51.016983 15912 ts_tablet_manager.cc:1939] T 61802bd1abff435c807ce780327af458 P 7ee10eeddace4536ab3946483f356acb: Deleting consensus metadata
I20260806 21:15:51.018420 15124 catalog_manager.cc:4841] TS 7ee10eeddace4536ab3946483f356acb (127.16.178.3:34927): tablet 61802bd1abff435c807ce780327af458 (table test-table [id=c23f5b30dec64ac6b48fd789144b32e6]) successfully deleted
I20260806 21:15:51.018518 15126 catalog_manager.cc:4841] TS c2357e255f58455391ae1d8df887f205 (127.16.178.2:32945): tablet 61802bd1abff435c807ce780327af458 (table test-table [id=c23f5b30dec64ac6b48fd789144b32e6]) successfully deleted
W20260806 21:15:51.021539 15126 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.178.1:38751 (error 108)
I20260806 21:15:51.021819 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 15434
I20260806 21:15:51.023051 15865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:51.022696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.1:38751 (local address 127.0.0.1:37934)
0806 21:15:51.022864 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 21:15:51.022951 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.1:38751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 21:15:51.023243 15126 catalog_manager.cc:4584] TS ba056c1a4b694e5995db1f06ff2d06af (127.16.178.1:38751): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 61802bd1abff435c807ce780327af458: Network error: Client connection negotiation failed: client connection to 127.16.178.1:38751: connect: Connection refused (error 111)
I20260806 21:15:51.037791 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 15635
I20260806 21:15:51.057828 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 15101
2026-08-06T21:15:51Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1249 ms)
[----------] 6 tests from EncryptionPolicyTest (11275 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050686797304-17096-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:15:51 dist-test-slave-lgr5 krb5kdc[15921](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:15:51 dist-test-slave-lgr5 krb5kdc[15921](info): set up 2 sockets
Aug 06 21:15:51 dist-test-slave-lgr5 krb5kdc[15921](info): commencing operation
krb5kdc: starting...
W20260806 21:15:52.232755 17096 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Aug 06 21:15:52 dist-test-slave-lgr5 krb5kdc[15921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050952, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:15:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:15:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050686797304-17096-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050686797304-17096-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 21:15:52.363147 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39759
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050686797304-17096-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39759
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:15:52.485504 15941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:15:52.485841 15941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:15:52.485908 15941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:15:52.490027 15941 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:15:52.490104 15941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:15:52.490131 15941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:15:52.490152 15941 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:15:52.490173 15941 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:15:52.495744 15941 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39759
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050686797304-17096-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39759
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15941
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:15:52.496963 15941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:15:52.498039 15941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:15:52.506844 15946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:52.506851 15955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:52.506932 15941 server_base.cc:1034] running on GCE node
W20260806 21:15:52.507107 15947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:52.507766 15941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:15:52.509006 15941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:15:52.510200 15941 hybrid_clock.cc:648] HybridClock initialized: now 1786050952510169 us; error 41 us; skew 500 ppm
Aug 06 21:15:52 dist-test-slave-lgr5 krb5kdc[15921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050952, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:15:52.513180 15941 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 21:15:52.514196 15941 webserver.cc:460] Webserver started at http://127.0.0.1:41433/ using document root <none> and password file <none>
I20260806 21:15:52.514750 15941 fs_manager.cc:359] Metadata directory not provided
I20260806 21:15:52.514808 15941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:15:52.515013 15941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:15:52.517374 15941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "596a7f37d1834316bcad0c0882b953ee"
format_stamp: "Formatted at 2026-08-06 21:15:52 on dist-test-slave-lgr5"
server_key: "f17f5b90b2bef3cc2f4e6c77ecfe01c8"
server_key_iv: "bb267c70ff93cc63efc6de974f0cf3cd"
server_key_version: "encryptionkey@0"
I20260806 21:15:52.517977 15941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "596a7f37d1834316bcad0c0882b953ee"
format_stamp: "Formatted at 2026-08-06 21:15:52 on dist-test-slave-lgr5"
server_key: "f17f5b90b2bef3cc2f4e6c77ecfe01c8"
server_key_iv: "bb267c70ff93cc63efc6de974f0cf3cd"
server_key_version: "encryptionkey@0"
I20260806 21:15:52.522174 15941 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:15:52.524767 15965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:52.526039 15941 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 21:15:52.526157 15941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "596a7f37d1834316bcad0c0882b953ee"
format_stamp: "Formatted at 2026-08-06 21:15:52 on dist-test-slave-lgr5"
server_key: "f17f5b90b2bef3cc2f4e6c77ecfe01c8"
server_key_iv: "bb267c70ff93cc63efc6de974f0cf3cd"
server_key_version: "encryptionkey@0"
I20260806 21:15:52.526278 15941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:15:52.550882 15941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:15:52.554167 15941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:15:52.554365 15941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:15:52.561636 15941 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39759
I20260806 21:15:52.561653 16038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39759 every 8 connection(s)
I20260806 21:15:52.562734 15941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:15:52.565918 16039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:15:52.571399 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 15941
I20260806 21:15:52.571565 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:15:52.571851 17096 external_mini_cluster.cc:1428] Setting key db5571ba9894d9e60564465dc6d42be2
I20260806 21:15:52.572724 16039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 596a7f37d1834316bcad0c0882b953ee: Bootstrap starting.
I20260806 21:15:52.575637 16039 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 596a7f37d1834316bcad0c0882b953ee: Neither blocks nor log segments found. Creating new log.
I20260806 21:15:52.576480 16039 log.cc:824] T 00000000000000000000000000000000 P 596a7f37d1834316bcad0c0882b953ee: Log is configured to *not* fsync() on all Append() calls
I20260806 21:15:52.578508 16039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 596a7f37d1834316bcad0c0882b953ee: No bootstrap required, opened a new log
Aug 06 21:15:52 dist-test-slave-lgr5 krb5kdc[15921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050952, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 21:15:52.582899 16039 raft_consensus.cc:348] T 00000000000000000000000000000000 P 596a7f37d1834316bcad0c0882b953ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "596a7f37d1834316bcad0c0882b953ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39759 } }
I20260806 21:15:52.583217 16039 raft_consensus.cc:374] T 00000000000000000000000000000000 P 596a7f37d1834316bcad0c0882b953ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:15:52.583300 16039 raft_consensus.cc:729] T 00000000000000000000000000000000 P 596a7f37d1834316bcad0c0882b953ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 596a7f37d1834316bcad0c0882b953ee, State: Initialized, Role: FOLLOWER
I20260806 21:15:52.583773 16039 consensus_queue.cc:260] T 00000000000000000000000000000000 P 596a7f37d1834316bcad0c0882b953ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "596a7f37d1834316bcad0c0882b953ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39759 } }
I20260806 21:15:52.583930 16039 raft_consensus.cc:388] T 00000000000000000000000000000000 P 596a7f37d1834316bcad0c0882b953ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:15:52.583997 16039 raft_consensus.cc:482] T 00000000000000000000000000000000 P 596a7f37d1834316bcad0c0882b953ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:15:52.584118 16039 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 596a7f37d1834316bcad0c0882b953ee [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:15:52.585095 16039 raft_consensus.cc:504] T 00000000000000000000000000000000 P 596a7f37d1834316bcad0c0882b953ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "596a7f37d1834316bcad0c0882b953ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39759 } }
I20260806 21:15:52.585546 16039 leader_election.cc:302] T 00000000000000000000000000000000 P 596a7f37d1834316bcad0c0882b953ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 596a7f37d1834316bcad0c0882b953ee; no voters:
I20260806 21:15:52.585940 16039 leader_election.cc:288] T 00000000000000000000000000000000 P 596a7f37d1834316bcad0c0882b953ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:15:52.586091 16044 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 596a7f37d1834316bcad0c0882b953ee [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:15:52.586340 16044 raft_consensus.cc:686] T 00000000000000000000000000000000 P 596a7f37d1834316bcad0c0882b953ee [term 1 LEADER]: Becoming Leader. State: Replica: 596a7f37d1834316bcad0c0882b953ee, State: Running, Role: LEADER
I20260806 21:15:52.586755 16044 consensus_queue.cc:237] T 00000000000000000000000000000000 P 596a7f37d1834316bcad0c0882b953ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "596a7f37d1834316bcad0c0882b953ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39759 } }
I20260806 21:15:52.587445 16039 sys_catalog.cc:564] T 00000000000000000000000000000000 P 596a7f37d1834316bcad0c0882b953ee [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:15:52.588707 16046 sys_catalog.cc:455] T 00000000000000000000000000000000 P 596a7f37d1834316bcad0c0882b953ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 596a7f37d1834316bcad0c0882b953ee. Latest consensus state: current_term: 1 leader_uuid: "596a7f37d1834316bcad0c0882b953ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "596a7f37d1834316bcad0c0882b953ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39759 } } }
I20260806 21:15:52.588853 16046 sys_catalog.cc:458] T 00000000000000000000000000000000 P 596a7f37d1834316bcad0c0882b953ee [sys.catalog]: This master's current role is: LEADER
I20260806 21:15:52.589217 16045 sys_catalog.cc:455] T 00000000000000000000000000000000 P 596a7f37d1834316bcad0c0882b953ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "596a7f37d1834316bcad0c0882b953ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "596a7f37d1834316bcad0c0882b953ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39759 } } }
I20260806 21:15:52.589289 16060 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:15:52.589406 16045 sys_catalog.cc:458] T 00000000000000000000000000000000 P 596a7f37d1834316bcad0c0882b953ee [sys.catalog]: This master's current role is: LEADER
I20260806 21:15:52.589629 16042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:52.573376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47661 (local address 127.0.0.1:39759)
0806 21:15:52.573823 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:52.573855 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:52.573893 (+ 38us) server_negotiation.cc:407] Connection header received
0806 21:15:52.574611 (+ 718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:52.574637 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:52.575142 (+ 505us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:52.575566 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:15:52.576569 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:52.577580 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:52.578440 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:52.578963 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:52.579128 (+ 165us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:52.582002 (+ 2874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:15:52.582008 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:15:52.582023 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:15:52.582055 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:15:52.584199 (+ 2144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:52.584870 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:52.584876 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:52.584881 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:52.585015 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:52.585355 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:52.585359 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:52.585361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:52.585929 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:15:52.586185 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:52.586812 (+ 627us) server_negotiation.cc:299] Negotiation successful
0806 21:15:52.587071 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":132,"threads_started":1}
I20260806 21:15:52.593717 17096 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:15:52.593994 16060 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:15:52.600296 16060 catalog_manager.cc:1347] Generated new cluster ID: 1fe3391f49c74449ab6b69e31f480938
I20260806 21:15:52.600382 16060 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:15:52.604965 16042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:52.596306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53857 (local address 127.0.0.1:39759)
0806 21:15:52.596437 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:52.596443 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:52.596742 (+ 299us) server_negotiation.cc:407] Connection header received
0806 21:15:52.596871 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:52.596873 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:52.596931 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:52.597016 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:15:52.597888 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:52.598445 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:52.599539 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:52.599922 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:52.600028 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:52.601700 (+ 1672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:15:52.601703 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:15:52.601705 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:15:52.601730 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:15:52.603059 (+ 1329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:52.603539 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:52.603542 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:52.603543 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:52.603622 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:52.603935 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:52.603938 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:52.603939 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:52.604173 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:15:52.604331 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:52.604691 (+ 360us) server_negotiation.cc:299] Negotiation successful
0806 21:15:52.604786 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 21:15:52.616343 16060 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:15:52.617802 16060 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:15:52.628767 16060 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 596a7f37d1834316bcad0c0882b953ee: Generated new TSK 0
I20260806 21:15:52.629731 16060 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:15:52.635309 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 15941
W20260806 21:15:52.654945 15917 connection.cc:535] client connection to 127.0.0.1:39759 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39759): Connection reset by peer (error 104)
2026-08-06T21:15:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1575 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050686797304-17096-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:15:52 dist-test-slave-lgr5 krb5kdc[16089](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:15:52 dist-test-slave-lgr5 krb5kdc[16089](info): set up 2 sockets
Aug 06 21:15:52 dist-test-slave-lgr5 krb5kdc[16089](info): commencing operation
krb5kdc: starting...
W20260806 21:15:53.792928 17096 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Aug 06 21:15:53 dist-test-slave-lgr5 krb5kdc[16089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:15:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:15:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050686797304-17096-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050686797304-17096-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 21:15:53.927772 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45997
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050686797304-17096-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45997
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:15:54.048198 16112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:15:54.048516 16112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:15:54.048558 16112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:15:54.052927 16112 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:15:54.053021 16112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:15:54.053040 16112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:15:54.053057 16112 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:15:54.053072 16112 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:15:54.058851 16112 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45997
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050686797304-17096-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16112
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:15:54.060117 16112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:15:54.061244 16112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:15:54.070269 16126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:54.070307 16128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:54.070436 16112 server_base.cc:1034] running on GCE node
W20260806 21:15:54.070307 16124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:54.071264 16112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:15:54.072603 16112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:15:54.073804 16112 hybrid_clock.cc:648] HybridClock initialized: now 1786050954073777 us; error 46 us; skew 500 ppm
Aug 06 21:15:54 dist-test-slave-lgr5 krb5kdc[16089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050954, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:15:54.077065 16112 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 21:15:54.078281 16112 webserver.cc:460] Webserver started at http://127.0.0.1:35177/ using document root <none> and password file <none>
I20260806 21:15:54.078990 16112 fs_manager.cc:359] Metadata directory not provided
I20260806 21:15:54.079087 16112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:15:54.079351 16112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:15:54.081925 16112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "0e6446880ed646e8ad792f26f1951b64"
format_stamp: "Formatted at 2026-08-06 21:15:54 on dist-test-slave-lgr5"
server_key: "2a934cfd12895cf5c3555b38d4672a9b"
server_key_iv: "e6e39cf2c31e3a5e273cddf41c952699"
server_key_version: "encryptionkey@0"
I20260806 21:15:54.082710 16112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "0e6446880ed646e8ad792f26f1951b64"
format_stamp: "Formatted at 2026-08-06 21:15:54 on dist-test-slave-lgr5"
server_key: "2a934cfd12895cf5c3555b38d4672a9b"
server_key_iv: "e6e39cf2c31e3a5e273cddf41c952699"
server_key_version: "encryptionkey@0"
I20260806 21:15:54.087720 16112 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 21:15:54.090684 16139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:54.092216 16112 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:15:54.092360 16112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "0e6446880ed646e8ad792f26f1951b64"
format_stamp: "Formatted at 2026-08-06 21:15:54 on dist-test-slave-lgr5"
server_key: "2a934cfd12895cf5c3555b38d4672a9b"
server_key_iv: "e6e39cf2c31e3a5e273cddf41c952699"
server_key_version: "encryptionkey@0"
I20260806 21:15:54.092489 16112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:15:54.105073 16112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:15:54.108418 16112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:15:54.108644 16112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:15:54.116093 16112 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45997
I20260806 21:15:54.116159 16222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45997 every 8 connection(s)
I20260806 21:15:54.117153 16112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:15:54.120230 16224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:15:54.125907 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 16112
I20260806 21:15:54.126071 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:15:54.126408 17096 external_mini_cluster.cc:1428] Setting key 00b966d738a376dfe97f7112fe4d00b1
I20260806 21:15:54.126493 16224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e6446880ed646e8ad792f26f1951b64: Bootstrap starting.
I20260806 21:15:54.130729 16224 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e6446880ed646e8ad792f26f1951b64: Neither blocks nor log segments found. Creating new log.
I20260806 21:15:54.131794 16224 log.cc:824] T 00000000000000000000000000000000 P 0e6446880ed646e8ad792f26f1951b64: Log is configured to *not* fsync() on all Append() calls
I20260806 21:15:54.134217 16224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e6446880ed646e8ad792f26f1951b64: No bootstrap required, opened a new log
I20260806 21:15:54.137287 16228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:54.127911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37079 (local address 127.0.0.1:45997)
0806 21:15:54.128582 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:54.128602 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:54.128641 (+ 39us) server_negotiation.cc:407] Connection header received
0806 21:15:54.129455 (+ 814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:54.129475 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:54.129936 (+ 461us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:54.130298 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:15:54.130962 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:54.132129 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:54.133231 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:54.133738 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:54.133872 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:54.134959 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:15:54.134964 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:15:54.134995 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:15:54.135457 (+ 462us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:15:54.135517 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:15:54.135543 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:15:54.135649 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:54.135850 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 21:15:54.136012 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":152,"threads_started":1}
I20260806 21:15:54.137843 16224 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e6446880ed646e8ad792f26f1951b64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e6446880ed646e8ad792f26f1951b64" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45997 } }
I20260806 21:15:54.138100 16224 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0e6446880ed646e8ad792f26f1951b64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:15:54.138139 16224 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e6446880ed646e8ad792f26f1951b64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e6446880ed646e8ad792f26f1951b64, State: Initialized, Role: FOLLOWER
I20260806 21:15:54.138622 16224 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e6446880ed646e8ad792f26f1951b64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e6446880ed646e8ad792f26f1951b64" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45997 } }
I20260806 21:15:54.138792 16224 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e6446880ed646e8ad792f26f1951b64 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:15:54.138854 16224 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e6446880ed646e8ad792f26f1951b64 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:15:54.138957 16224 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e6446880ed646e8ad792f26f1951b64 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:15:54.140060 16224 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e6446880ed646e8ad792f26f1951b64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e6446880ed646e8ad792f26f1951b64" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45997 } }
I20260806 21:15:54.140453 16224 leader_election.cc:302] T 00000000000000000000000000000000 P 0e6446880ed646e8ad792f26f1951b64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e6446880ed646e8ad792f26f1951b64; no voters:
I20260806 21:15:54.140705 17096 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:15:54.140825 16224 leader_election.cc:288] T 00000000000000000000000000000000 P 0e6446880ed646e8ad792f26f1951b64 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:15:54.140940 16230 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e6446880ed646e8ad792f26f1951b64 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:15:54.141197 16230 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e6446880ed646e8ad792f26f1951b64 [term 1 LEADER]: Becoming Leader. State: Replica: 0e6446880ed646e8ad792f26f1951b64, State: Running, Role: LEADER
I20260806 21:15:54.141572 16230 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e6446880ed646e8ad792f26f1951b64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e6446880ed646e8ad792f26f1951b64" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45997 } }
I20260806 21:15:54.142266 16224 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e6446880ed646e8ad792f26f1951b64 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:15:54.142867 16230 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e6446880ed646e8ad792f26f1951b64 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e6446880ed646e8ad792f26f1951b64. Latest consensus state: current_term: 1 leader_uuid: "0e6446880ed646e8ad792f26f1951b64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e6446880ed646e8ad792f26f1951b64" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45997 } } }
I20260806 21:15:54.143000 16230 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e6446880ed646e8ad792f26f1951b64 [sys.catalog]: This master's current role is: LEADER
I20260806 21:15:54.143569 16232 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e6446880ed646e8ad792f26f1951b64 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e6446880ed646e8ad792f26f1951b64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e6446880ed646e8ad792f26f1951b64" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45997 } } }
I20260806 21:15:54.143669 16232 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e6446880ed646e8ad792f26f1951b64 [sys.catalog]: This master's current role is: LEADER
I20260806 21:15:54.144265 16240 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:15:54.147716 16240 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:15:54.148591 16228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:54.143573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49397 (local address 127.0.0.1:45997)
0806 21:15:54.143689 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:54.143695 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:54.143848 (+ 153us) server_negotiation.cc:407] Connection header received
0806 21:15:54.144061 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:54.144064 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:54.144147 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:54.144204 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:15:54.144661 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:54.145430 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:54.146554 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:54.146891 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:54.147002 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:54.148017 (+ 1015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:15:54.148021 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:15:54.148025 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:15:54.148041 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:15:54.148073 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:15:54.148115 (+ 42us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:15:54.148208 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:54.148407 (+ 199us) server_negotiation.cc:299] Negotiation successful
0806 21:15:54.148445 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":45}
I20260806 21:15:54.157312 16240 catalog_manager.cc:1347] Generated new cluster ID: b454f13ac0b148d1ad8f7bfa8f90acb5
I20260806 21:15:54.157394 16240 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:15:54.184178 16240 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:15:54.185217 16240 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:15:54.197086 16240 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0e6446880ed646e8ad792f26f1951b64: Generated new TSK 0
I20260806 21:15:54.197994 16240 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:15:54.203047 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 16112
W20260806 21:15:54.214699 16086 connection.cc:535] client connection to 127.0.0.1:45997 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45997): Connection reset by peer (error 104)
2026-08-06T21:15:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1559 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050686797304-17096-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:15:54 dist-test-slave-lgr5 krb5kdc[16269](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:15:54 dist-test-slave-lgr5 krb5kdc[16269](info): set up 2 sockets
Aug 06 21:15:54 dist-test-slave-lgr5 krb5kdc[16269](info): commencing operation
krb5kdc: starting...
W20260806 21:15:55.360890 17096 mini_kdc.cc:121] Time spent starting KDC: real 1.132s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Aug 06 21:15:55 dist-test-slave-lgr5 krb5kdc[16269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050955, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:15:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:15:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050686797304-17096-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050686797304-17096-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 21:15:55.495085 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35497
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050686797304-17096-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35497
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:15:55.610844 16286 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:15:55.611171 16286 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:15:55.611232 16286 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:15:55.615455 16286 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:15:55.615542 16286 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:15:55.615569 16286 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:15:55.615590 16286 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:15:55.615620 16286 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:15:55.621279 16286 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35497
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050686797304-17096-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16286
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:15:55.622493 16286 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:15:55.623569 16286 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:15:55.632495 16295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:55.632495 16298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:55.632488 16296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:55.632853 16286 server_base.cc:1034] running on GCE node
I20260806 21:15:55.633472 16286 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:15:55.634683 16286 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:15:55.635879 16286 hybrid_clock.cc:648] HybridClock initialized: now 1786050955635863 us; error 31 us; skew 500 ppm
Aug 06 21:15:55 dist-test-slave-lgr5 krb5kdc[16269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050955, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:15:55.642722 16286 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 21:15:55.643801 16286 webserver.cc:460] Webserver started at http://127.0.0.1:40727/ using document root <none> and password file <none>
I20260806 21:15:55.644461 16286 fs_manager.cc:359] Metadata directory not provided
I20260806 21:15:55.644534 16286 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:15:55.644726 16286 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:15:55.647006 16286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "a1cda03368f44698b2f6d1ac29f87bd2"
format_stamp: "Formatted at 2026-08-06 21:15:55 on dist-test-slave-lgr5"
server_key: "d26300593b7325cad455ea1c059ec1da"
server_key_iv: "5b1748908d1762d3e12a930e72015b26"
server_key_version: "encryptionkey@0"
I20260806 21:15:55.647663 16286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "a1cda03368f44698b2f6d1ac29f87bd2"
format_stamp: "Formatted at 2026-08-06 21:15:55 on dist-test-slave-lgr5"
server_key: "d26300593b7325cad455ea1c059ec1da"
server_key_iv: "5b1748908d1762d3e12a930e72015b26"
server_key_version: "encryptionkey@0"
I20260806 21:15:55.651878 16286 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260806 21:15:55.654526 16309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:55.655821 16286 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 21:15:55.655936 16286 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "a1cda03368f44698b2f6d1ac29f87bd2"
format_stamp: "Formatted at 2026-08-06 21:15:55 on dist-test-slave-lgr5"
server_key: "d26300593b7325cad455ea1c059ec1da"
server_key_iv: "5b1748908d1762d3e12a930e72015b26"
server_key_version: "encryptionkey@0"
I20260806 21:15:55.656036 16286 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:15:55.660761 16286 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:15:55.664209 16286 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:15:55.664386 16286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:15:55.671911 16286 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35497
I20260806 21:15:55.671967 16393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35497 every 8 connection(s)
I20260806 21:15:55.672978 16286 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:15:55.675879 16394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:15:55.681774 16394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1cda03368f44698b2f6d1ac29f87bd2: Bootstrap starting.
I20260806 21:15:55.682610 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 16286
I20260806 21:15:55.682763 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:15:55.683084 17096 external_mini_cluster.cc:1428] Setting key f8492a7311590fe0fe7fc0362fb4ebf0
I20260806 21:15:55.684993 16394 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1cda03368f44698b2f6d1ac29f87bd2: Neither blocks nor log segments found. Creating new log.
I20260806 21:15:55.685771 16394 log.cc:824] T 00000000000000000000000000000000 P a1cda03368f44698b2f6d1ac29f87bd2: Log is configured to *not* fsync() on all Append() calls
I20260806 21:15:55.688158 16394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1cda03368f44698b2f6d1ac29f87bd2: No bootstrap required, opened a new log
I20260806 21:15:55.688598 16397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:55.684430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58505 (local address 127.0.0.1:35497)
0806 21:15:55.684935 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:55.684950 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:55.685000 (+ 50us) server_negotiation.cc:407] Connection header received
0806 21:15:55.685729 (+ 729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:55.685749 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:55.686083 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:55.686380 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:15:55.687143 (+ 763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:15:55.687149 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:15:55.687164 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:15:55.687176 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:15:55.687240 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:15:55.687267 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:15:55.687388 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:55.687455 (+ 67us) server_negotiation.cc:299] Negotiation successful
0806 21:15:55.687613 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":157,"threads_started":1}
I20260806 21:15:55.691570 17096 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:15:55.692498 16394 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1cda03368f44698b2f6d1ac29f87bd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1cda03368f44698b2f6d1ac29f87bd2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35497 } }
I20260806 21:15:55.692736 16394 raft_consensus.cc:374] T 00000000000000000000000000000000 P a1cda03368f44698b2f6d1ac29f87bd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:15:55.692816 16394 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1cda03368f44698b2f6d1ac29f87bd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1cda03368f44698b2f6d1ac29f87bd2, State: Initialized, Role: FOLLOWER
I20260806 21:15:55.693254 16394 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1cda03368f44698b2f6d1ac29f87bd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1cda03368f44698b2f6d1ac29f87bd2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35497 } }
I20260806 21:15:55.693392 16394 raft_consensus.cc:388] T 00000000000000000000000000000000 P a1cda03368f44698b2f6d1ac29f87bd2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:15:55.693457 16394 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1cda03368f44698b2f6d1ac29f87bd2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:15:55.693563 16394 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1cda03368f44698b2f6d1ac29f87bd2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:15:55.694553 16394 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1cda03368f44698b2f6d1ac29f87bd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1cda03368f44698b2f6d1ac29f87bd2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35497 } }
I20260806 21:15:55.694888 16394 leader_election.cc:302] T 00000000000000000000000000000000 P a1cda03368f44698b2f6d1ac29f87bd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1cda03368f44698b2f6d1ac29f87bd2; no voters:
I20260806 21:15:55.695237 16394 leader_election.cc:288] T 00000000000000000000000000000000 P a1cda03368f44698b2f6d1ac29f87bd2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:15:55.695338 16412 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1cda03368f44698b2f6d1ac29f87bd2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:15:55.695444 16397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:55.693697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54987 (local address 127.0.0.1:35497)
0806 21:15:55.693826 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:55.693831 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:55.694036 (+ 205us) server_negotiation.cc:407] Connection header received
0806 21:15:55.694143 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:55.694146 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:55.694181 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:55.694245 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:15:55.694929 (+ 684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:15:55.694939 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:15:55.694942 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:15:55.694954 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:15:55.694981 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:15:55.694992 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:15:55.695082 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:55.695212 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 21:15:55.695250 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 21:15:55.695631 16412 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1cda03368f44698b2f6d1ac29f87bd2 [term 1 LEADER]: Becoming Leader. State: Replica: a1cda03368f44698b2f6d1ac29f87bd2, State: Running, Role: LEADER
I20260806 21:15:55.695956 16412 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1cda03368f44698b2f6d1ac29f87bd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1cda03368f44698b2f6d1ac29f87bd2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35497 } }
I20260806 21:15:55.696732 16394 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1cda03368f44698b2f6d1ac29f87bd2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:15:55.697232 16413 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1cda03368f44698b2f6d1ac29f87bd2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1cda03368f44698b2f6d1ac29f87bd2. Latest consensus state: current_term: 1 leader_uuid: "a1cda03368f44698b2f6d1ac29f87bd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1cda03368f44698b2f6d1ac29f87bd2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35497 } } }
I20260806 21:15:55.697355 16413 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1cda03368f44698b2f6d1ac29f87bd2 [sys.catalog]: This master's current role is: LEADER
I20260806 21:15:55.697700 16415 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1cda03368f44698b2f6d1ac29f87bd2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a1cda03368f44698b2f6d1ac29f87bd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1cda03368f44698b2f6d1ac29f87bd2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35497 } } }
I20260806 21:15:55.697794 16415 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1cda03368f44698b2f6d1ac29f87bd2 [sys.catalog]: This master's current role is: LEADER
I20260806 21:15:55.697806 16421 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:15:55.700800 16421 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:15:55.706218 16421 catalog_manager.cc:1347] Generated new cluster ID: cbfc054ea78e4bc6bda9b3de0042ac3d
I20260806 21:15:55.706310 16421 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:15:55.721437 16421 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:15:55.721836 16421 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:15:55.732589 16421 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a1cda03368f44698b2f6d1ac29f87bd2: Generated new TSK 0
I20260806 21:15:55.733371 16421 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:15:55.745455 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 16286
2026-08-06T21:15:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1550 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050686797304-17096-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:15:55 dist-test-slave-lgr5 krb5kdc[16444](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:15:55 dist-test-slave-lgr5 krb5kdc[16444](info): set up 2 sockets
Aug 06 21:15:55 dist-test-slave-lgr5 krb5kdc[16444](info): commencing operation
krb5kdc: starting...
W20260806 21:15:56.913518 17096 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Aug 06 21:15:57 dist-test-slave-lgr5 krb5kdc[16444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050956, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:15:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:15:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050686797304-17096-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050686797304-17096-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 21:15:57.043054 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43787
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050686797304-17096-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43787
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:15:57.159365 16462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:15:57.159669 16462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:15:57.159713 16462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:15:57.164394 16462 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:15:57.164484 16462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:15:57.164503 16462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:15:57.164518 16462 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:15:57.164533 16462 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:15:57.170325 16462 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43787
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050686797304-17096-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16462
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:15:57.171521 16462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:15:57.172690 16462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:15:57.182020 16470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:57.182031 16468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:57.182020 16472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:57.182492 16462 server_base.cc:1034] running on GCE node
I20260806 21:15:57.183138 16462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:15:57.184401 16462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:15:57.185570 16462 hybrid_clock.cc:648] HybridClock initialized: now 1786050957185547 us; error 41 us; skew 500 ppm
Aug 06 21:15:57 dist-test-slave-lgr5 krb5kdc[16444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050957, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:15:57.188540 16462 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 21:15:57.189525 16462 webserver.cc:460] Webserver started at http://127.0.0.1:44825/ using document root <none> and password file <none>
I20260806 21:15:57.190119 16462 fs_manager.cc:359] Metadata directory not provided
I20260806 21:15:57.190184 16462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:15:57.190409 16462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:15:57.192655 16462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "9153b9f459be462fb9d03225b2874f10"
format_stamp: "Formatted at 2026-08-06 21:15:57 on dist-test-slave-lgr5"
server_key: "43802a6dcffb0d40e04cbe1aa1e146af"
server_key_iv: "b4521afb2772a62145c66cfbdc7a6e77"
server_key_version: "encryptionkey@0"
I20260806 21:15:57.193246 16462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "9153b9f459be462fb9d03225b2874f10"
format_stamp: "Formatted at 2026-08-06 21:15:57 on dist-test-slave-lgr5"
server_key: "43802a6dcffb0d40e04cbe1aa1e146af"
server_key_iv: "b4521afb2772a62145c66cfbdc7a6e77"
server_key_version: "encryptionkey@0"
I20260806 21:15:57.197466 16462 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:15:57.199951 16481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:57.201238 16462 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 21:15:57.201365 16462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "9153b9f459be462fb9d03225b2874f10"
format_stamp: "Formatted at 2026-08-06 21:15:57 on dist-test-slave-lgr5"
server_key: "43802a6dcffb0d40e04cbe1aa1e146af"
server_key_iv: "b4521afb2772a62145c66cfbdc7a6e77"
server_key_version: "encryptionkey@0"
I20260806 21:15:57.201481 16462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:15:57.223670 16462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:15:57.227174 16462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:15:57.227347 16462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:15:57.235144 16462 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43787
I20260806 21:15:57.235162 16565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43787 every 8 connection(s)
I20260806 21:15:57.236367 16462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:15:57.239348 16566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:15:57.240645 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 16462
I20260806 21:15:57.240763 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:15:57.241091 17096 external_mini_cluster.cc:1428] Setting key 69aa0047e5d1276aca6694308bcb6c85
I20260806 21:15:57.246129 16566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9153b9f459be462fb9d03225b2874f10: Bootstrap starting.
I20260806 21:15:57.248833 16566 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9153b9f459be462fb9d03225b2874f10: Neither blocks nor log segments found. Creating new log.
I20260806 21:15:57.249771 16566 log.cc:824] T 00000000000000000000000000000000 P 9153b9f459be462fb9d03225b2874f10: Log is configured to *not* fsync() on all Append() calls
Aug 06 21:15:57 dist-test-slave-lgr5 krb5kdc[16444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050956, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 21:15:57.251842 16566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9153b9f459be462fb9d03225b2874f10: No bootstrap required, opened a new log
I20260806 21:15:57.255985 16570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:57.242642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46063 (local address 127.0.0.1:43787)
0806 21:15:57.243099 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:57.243114 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:57.243152 (+ 38us) server_negotiation.cc:407] Connection header received
0806 21:15:57.243786 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:57.243802 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:57.244185 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:57.244478 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:15:57.245354 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:57.246376 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:57.247518 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:57.248001 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:57.248161 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:57.250911 (+ 2750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:15:57.250916 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:15:57.250931 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:15:57.250959 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:15:57.252600 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:57.253061 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:57.253085 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:57.253092 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:57.253235 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:57.253593 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:57.253597 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:57.253599 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:57.254163 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:15:57.254362 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:57.254667 (+ 305us) server_negotiation.cc:299] Negotiation successful
0806 21:15:57.254987 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":139,"threads_started":1}
I20260806 21:15:57.256064 16566 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9153b9f459be462fb9d03225b2874f10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9153b9f459be462fb9d03225b2874f10" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43787 } }
I20260806 21:15:57.256350 16566 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9153b9f459be462fb9d03225b2874f10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:15:57.256412 16566 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9153b9f459be462fb9d03225b2874f10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9153b9f459be462fb9d03225b2874f10, State: Initialized, Role: FOLLOWER
I20260806 21:15:57.256866 16566 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9153b9f459be462fb9d03225b2874f10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9153b9f459be462fb9d03225b2874f10" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43787 } }
I20260806 21:15:57.256999 16566 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9153b9f459be462fb9d03225b2874f10 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:15:57.257063 16566 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9153b9f459be462fb9d03225b2874f10 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:15:57.257161 16566 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9153b9f459be462fb9d03225b2874f10 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:15:57.258265 16566 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9153b9f459be462fb9d03225b2874f10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9153b9f459be462fb9d03225b2874f10" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43787 } }
I20260806 21:15:57.258626 16566 leader_election.cc:302] T 00000000000000000000000000000000 P 9153b9f459be462fb9d03225b2874f10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9153b9f459be462fb9d03225b2874f10; no voters:
I20260806 21:15:57.258965 16566 leader_election.cc:288] T 00000000000000000000000000000000 P 9153b9f459be462fb9d03225b2874f10 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:15:57.258983 17096 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:15:57.259090 16576 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9153b9f459be462fb9d03225b2874f10 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:15:57.259305 16576 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9153b9f459be462fb9d03225b2874f10 [term 1 LEADER]: Becoming Leader. State: Replica: 9153b9f459be462fb9d03225b2874f10, State: Running, Role: LEADER
I20260806 21:15:57.259642 16576 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9153b9f459be462fb9d03225b2874f10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9153b9f459be462fb9d03225b2874f10" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43787 } }
I20260806 21:15:57.260311 16566 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9153b9f459be462fb9d03225b2874f10 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:15:57.261179 16579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9153b9f459be462fb9d03225b2874f10 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9153b9f459be462fb9d03225b2874f10. Latest consensus state: current_term: 1 leader_uuid: "9153b9f459be462fb9d03225b2874f10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9153b9f459be462fb9d03225b2874f10" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43787 } } }
I20260806 21:15:57.261312 16579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9153b9f459be462fb9d03225b2874f10 [sys.catalog]: This master's current role is: LEADER
I20260806 21:15:57.261651 16577 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9153b9f459be462fb9d03225b2874f10 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9153b9f459be462fb9d03225b2874f10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9153b9f459be462fb9d03225b2874f10" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43787 } } }
I20260806 21:15:57.261745 16577 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9153b9f459be462fb9d03225b2874f10 [sys.catalog]: This master's current role is: LEADER
W20260806 21:15:57.265550 16608 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9153b9f459be462fb9d03225b2874f10: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 21:15:57.265643 16608 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 21:15:57.265726 16596 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:15:57.266629 16596 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:15:57.269965 16570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:57.261256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38497 (local address 127.0.0.1:43787)
0806 21:15:57.261365 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:57.261370 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:57.262753 (+ 1383us) server_negotiation.cc:407] Connection header received
0806 21:15:57.262859 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:57.262862 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:57.262901 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:57.262974 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:15:57.263636 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:57.264285 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:57.265133 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:57.265372 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:57.265461 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:57.266828 (+ 1367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:15:57.266832 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:15:57.266835 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:15:57.266861 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:15:57.268222 (+ 1361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:57.268614 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:57.268617 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:57.268619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:57.268710 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:15:57.269065 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:15:57.269068 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:15:57.269070 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:15:57.269285 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:15:57.269412 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:57.269685 (+ 273us) server_negotiation.cc:299] Negotiation successful
0806 21:15:57.269807 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 21:15:57.272686 16596 catalog_manager.cc:1347] Generated new cluster ID: a2b792cd894e4e95b57d8c8d7902f1a9
I20260806 21:15:57.272759 16596 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:15:57.298189 16596 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:15:57.299330 16596 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:15:57.305802 16596 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9153b9f459be462fb9d03225b2874f10: Generated new TSK 0
I20260806 21:15:57.306711 16596 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:15:57.322885 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 16462
2026-08-06T21:15:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1575 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050686797304-17096-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:15:57 dist-test-slave-lgr5 krb5kdc[16614](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:15:57 dist-test-slave-lgr5 krb5kdc[16614](info): set up 2 sockets
Aug 06 21:15:57 dist-test-slave-lgr5 krb5kdc[16614](info): commencing operation
krb5kdc: starting...
W20260806 21:15:58.479669 17096 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Aug 06 21:15:58 dist-test-slave-lgr5 krb5kdc[16614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050958, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:15:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:15:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050686797304-17096-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050686797304-17096-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 21:15:58.611233 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46753
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050686797304-17096-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46753
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:15:58.731132 16635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:15:58.731460 16635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:15:58.731514 16635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:15:58.735778 16635 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:15:58.735865 16635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:15:58.735884 16635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:15:58.735900 16635 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:15:58.735915 16635 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:15:58.741600 16635 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46753
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050686797304-17096-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:15:58.742683 16635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:15:58.743676 16635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:15:58.752061 16646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:58.752068 16650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:15:58.752121 16645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:15:58.752146 16635 server_base.cc:1034] running on GCE node
I20260806 21:15:58.752844 16635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:15:58.753924 16635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:15:58.755081 16635 hybrid_clock.cc:648] HybridClock initialized: now 1786050958755059 us; error 36 us; skew 500 ppm
Aug 06 21:15:58 dist-test-slave-lgr5 krb5kdc[16614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050958, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:15:58.757695 16635 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 21:15:58.758638 16635 webserver.cc:460] Webserver started at http://127.0.0.1:34905/ using document root <none> and password file <none>
I20260806 21:15:58.759184 16635 fs_manager.cc:359] Metadata directory not provided
I20260806 21:15:58.759244 16635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:15:58.759444 16635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:15:58.761523 16635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "1f6a41a854a3417fb972c666fbb0302d"
format_stamp: "Formatted at 2026-08-06 21:15:58 on dist-test-slave-lgr5"
server_key: "832351cc3091bb28f181abe90ac6291b"
server_key_iv: "215af8bb7e5f844126c82632f238fd78"
server_key_version: "encryptionkey@0"
I20260806 21:15:58.762071 16635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "1f6a41a854a3417fb972c666fbb0302d"
format_stamp: "Formatted at 2026-08-06 21:15:58 on dist-test-slave-lgr5"
server_key: "832351cc3091bb28f181abe90ac6291b"
server_key_iv: "215af8bb7e5f844126c82632f238fd78"
server_key_version: "encryptionkey@0"
I20260806 21:15:58.765766 16635 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 21:15:58.768261 16657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:15:58.769512 16635 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 21:15:58.769635 16635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "1f6a41a854a3417fb972c666fbb0302d"
format_stamp: "Formatted at 2026-08-06 21:15:58 on dist-test-slave-lgr5"
server_key: "832351cc3091bb28f181abe90ac6291b"
server_key_iv: "215af8bb7e5f844126c82632f238fd78"
server_key_version: "encryptionkey@0"
I20260806 21:15:58.769740 16635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:15:58.787740 16635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:15:58.790692 16635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:15:58.790892 16635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:15:58.798249 16635 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46753
I20260806 21:15:58.798272 16737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46753 every 8 connection(s)
I20260806 21:15:58.799409 16635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:15:58.802431 16738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:15:58.808387 16738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f6a41a854a3417fb972c666fbb0302d: Bootstrap starting.
I20260806 21:15:58.809314 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 16635
I20260806 21:15:58.809463 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:15:58.809725 17096 external_mini_cluster.cc:1428] Setting key a9097be61abb9102dbab81c320ec0331
I20260806 21:15:58.811292 16738 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f6a41a854a3417fb972c666fbb0302d: Neither blocks nor log segments found. Creating new log.
I20260806 21:15:58.812038 16738 log.cc:824] T 00000000000000000000000000000000 P 1f6a41a854a3417fb972c666fbb0302d: Log is configured to *not* fsync() on all Append() calls
I20260806 21:15:58.814040 16738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f6a41a854a3417fb972c666fbb0302d: No bootstrap required, opened a new log
I20260806 21:15:58.817983 16738 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f6a41a854a3417fb972c666fbb0302d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f6a41a854a3417fb972c666fbb0302d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46753 } }
I20260806 21:15:58.818254 16738 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f6a41a854a3417fb972c666fbb0302d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:15:58.818312 16738 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f6a41a854a3417fb972c666fbb0302d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f6a41a854a3417fb972c666fbb0302d, State: Initialized, Role: FOLLOWER
I20260806 21:15:58.818854 16742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:58.810916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43051 (local address 127.0.0.1:46753)
0806 21:15:58.811373 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:58.811391 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:58.811437 (+ 46us) server_negotiation.cc:407] Connection header received
0806 21:15:58.812136 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:58.812155 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:58.812527 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:58.812839 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:15:58.813296 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:58.814342 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:58.815238 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:58.815732 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:58.815884 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:58.816834 (+ 950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:15:58.816839 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:15:58.816853 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:15:58.817277 (+ 424us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:15:58.817336 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:15:58.817360 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:15:58.817454 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:58.817639 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 21:15:58.817916 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":126,"threads_started":1}
I20260806 21:15:58.818799 16738 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f6a41a854a3417fb972c666fbb0302d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f6a41a854a3417fb972c666fbb0302d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46753 } }
I20260806 21:15:58.819002 16738 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f6a41a854a3417fb972c666fbb0302d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:15:58.819084 16738 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f6a41a854a3417fb972c666fbb0302d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:15:58.819202 16738 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f6a41a854a3417fb972c666fbb0302d [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:15:58.820457 16738 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f6a41a854a3417fb972c666fbb0302d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f6a41a854a3417fb972c666fbb0302d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46753 } }
I20260806 21:15:58.820830 16738 leader_election.cc:302] T 00000000000000000000000000000000 P 1f6a41a854a3417fb972c666fbb0302d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f6a41a854a3417fb972c666fbb0302d; no voters:
I20260806 21:15:58.821245 16738 leader_election.cc:288] T 00000000000000000000000000000000 P 1f6a41a854a3417fb972c666fbb0302d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:15:58.821326 16745 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f6a41a854a3417fb972c666fbb0302d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:15:58.821597 16745 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f6a41a854a3417fb972c666fbb0302d [term 1 LEADER]: Becoming Leader. State: Replica: 1f6a41a854a3417fb972c666fbb0302d, State: Running, Role: LEADER
I20260806 21:15:58.822005 16745 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f6a41a854a3417fb972c666fbb0302d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f6a41a854a3417fb972c666fbb0302d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46753 } }
I20260806 21:15:58.822121 17096 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:15:58.822659 16738 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f6a41a854a3417fb972c666fbb0302d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:15:58.823889 16749 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f6a41a854a3417fb972c666fbb0302d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f6a41a854a3417fb972c666fbb0302d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f6a41a854a3417fb972c666fbb0302d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46753 } } }
I20260806 21:15:58.824069 16749 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f6a41a854a3417fb972c666fbb0302d [sys.catalog]: This master's current role is: LEADER
I20260806 21:15:58.823889 16750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f6a41a854a3417fb972c666fbb0302d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f6a41a854a3417fb972c666fbb0302d. Latest consensus state: current_term: 1 leader_uuid: "1f6a41a854a3417fb972c666fbb0302d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f6a41a854a3417fb972c666fbb0302d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46753 } } }
I20260806 21:15:58.824339 16750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f6a41a854a3417fb972c666fbb0302d [sys.catalog]: This master's current role is: LEADER
I20260806 21:15:58.825449 16771 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:15:58.829123 16771 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:15:58.829681 16742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:15:58.824710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48111 (local address 127.0.0.1:46753)
0806 21:15:58.824826 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0806 21:15:58.824830 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:15:58.825786 (+ 956us) server_negotiation.cc:407] Connection header received
0806 21:15:58.825923 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:15:58.825926 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:15:58.825989 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:15:58.826072 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:15:58.826512 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:58.827068 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:58.827973 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:15:58.828276 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:15:58.828371 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:15:58.829219 (+ 848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:15:58.829223 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:15:58.829225 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:15:58.829241 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:15:58.829271 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:15:58.829285 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:15:58.829350 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 21:15:58.829453 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 21:15:58.829545 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 21:15:58.835471 16771 catalog_manager.cc:1347] Generated new cluster ID: 4f8da767eb46479285e719572ed87ec9
I20260806 21:15:58.835553 16771 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:15:58.842624 16771 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:15:58.843580 16771 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:15:58.858212 16771 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f6a41a854a3417fb972c666fbb0302d: Generated new TSK 0
I20260806 21:15:58.858976 16771 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:15:58.864183 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 16635
2026-08-06T21:15:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1539 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050686797304-17096-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:15:58 dist-test-slave-lgr5 krb5kdc[16787](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:15:58 dist-test-slave-lgr5 krb5kdc[16787](info): set up 2 sockets
Aug 06 21:15:58 dist-test-slave-lgr5 krb5kdc[16787](info): commencing operation
krb5kdc: starting...
W20260806 21:16:00.023597 17096 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Aug 06 21:16:00 dist-test-slave-lgr5 krb5kdc[16787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050960, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:16:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:16:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050686797304-17096-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050686797304-17096-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 21:16:00.155792 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42605
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050686797304-17096-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42605
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:16:00.275660 16807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:16:00.275969 16807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:16:00.276011 16807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:16:00.280050 16807 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:16:00.280153 16807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:16:00.280172 16807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:16:00.280187 16807 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:16:00.280201 16807 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:16:00.285694 16807 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42605
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050686797304-17096-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16807
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:16:00.286864 16807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:16:00.287951 16807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:16:00.297716 16814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:16:00.297847 16807 server_base.cc:1034] running on GCE node
W20260806 21:16:00.297734 16815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:16:00.297744 16818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:16:00.298621 16807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:16:00.299851 16807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:16:00.301051 16807 hybrid_clock.cc:648] HybridClock initialized: now 1786050960300992 us; error 72 us; skew 500 ppm
Aug 06 21:16:00 dist-test-slave-lgr5 krb5kdc[16787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050960, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:16:00.304378 16807 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 21:16:00.305624 16807 webserver.cc:460] Webserver started at http://127.0.0.1:38515/ using document root <none> and password file <none>
I20260806 21:16:00.306332 16807 fs_manager.cc:359] Metadata directory not provided
I20260806 21:16:00.306421 16807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:16:00.306684 16807 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:16:00.309284 16807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "5226ad7169074565b377e7f7621a0915"
format_stamp: "Formatted at 2026-08-06 21:16:00 on dist-test-slave-lgr5"
server_key: "2de52aa951da3c3b37163c0d68c15d3b"
server_key_iv: "80714dc80c979926e7f7cf31f853df39"
server_key_version: "encryptionkey@0"
I20260806 21:16:00.309984 16807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "5226ad7169074565b377e7f7621a0915"
format_stamp: "Formatted at 2026-08-06 21:16:00 on dist-test-slave-lgr5"
server_key: "2de52aa951da3c3b37163c0d68c15d3b"
server_key_iv: "80714dc80c979926e7f7cf31f853df39"
server_key_version: "encryptionkey@0"
I20260806 21:16:00.314869 16807 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 21:16:00.318032 16832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:16:00.319594 16807 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:16:00.319779 16807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "5226ad7169074565b377e7f7621a0915"
format_stamp: "Formatted at 2026-08-06 21:16:00 on dist-test-slave-lgr5"
server_key: "2de52aa951da3c3b37163c0d68c15d3b"
server_key_iv: "80714dc80c979926e7f7cf31f853df39"
server_key_version: "encryptionkey@0"
I20260806 21:16:00.319914 16807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:16:00.325176 16807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:16:00.328531 16807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:16:00.328725 16807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:16:00.336055 16807 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42605
I20260806 21:16:00.336102 16911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42605 every 8 connection(s)
I20260806 21:16:00.337137 16807 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:16:00.340281 16912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:16:00.343375 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 16807
I20260806 21:16:00.343523 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:16:00.343791 17096 external_mini_cluster.cc:1428] Setting key 07cf00837bf016111d3c162742eb7711
I20260806 21:16:00.346552 16912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5226ad7169074565b377e7f7621a0915: Bootstrap starting.
I20260806 21:16:00.349272 16912 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5226ad7169074565b377e7f7621a0915: Neither blocks nor log segments found. Creating new log.
I20260806 21:16:00.349275 16915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:00.345220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33245 (local address 127.0.0.1:42605)
0806 21:16:00.345708 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:00.345725 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:00.345764 (+ 39us) server_negotiation.cc:407] Connection header received
0806 21:16:00.346409 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:00.346426 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:00.346749 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:00.347050 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:00.347914 (+ 864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:00.347919 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:00.347932 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:16:00.347944 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:00.348002 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:16:00.348026 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:00.348188 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:00.348286 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 21:16:00.348454 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":141,"threads_started":1}
I20260806 21:16:00.350260 16912 log.cc:824] T 00000000000000000000000000000000 P 5226ad7169074565b377e7f7621a0915: Log is configured to *not* fsync() on all Append() calls
I20260806 21:16:00.352819 16912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5226ad7169074565b377e7f7621a0915: No bootstrap required, opened a new log
I20260806 21:16:00.352897 17096 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:16:00.356976 16915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:00.355082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57319 (local address 127.0.0.1:42605)
0806 21:16:00.355221 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:00.355226 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:00.355528 (+ 302us) server_negotiation.cc:407] Connection header received
0806 21:16:00.355642 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:00.355644 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:00.355681 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:00.355764 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:00.356579 (+ 815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:00.356582 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:00.356584 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:16:00.356596 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:00.356624 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:16:00.356636 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:00.356720 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:00.356806 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 21:16:00.356840 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 21:16:00.357436 16912 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5226ad7169074565b377e7f7621a0915 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5226ad7169074565b377e7f7621a0915" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42605 } }
I20260806 21:16:00.357708 16912 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5226ad7169074565b377e7f7621a0915 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:16:00.357776 16912 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5226ad7169074565b377e7f7621a0915 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5226ad7169074565b377e7f7621a0915, State: Initialized, Role: FOLLOWER
I20260806 21:16:00.358263 16912 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5226ad7169074565b377e7f7621a0915 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5226ad7169074565b377e7f7621a0915" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42605 } }
I20260806 21:16:00.358405 16912 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5226ad7169074565b377e7f7621a0915 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:16:00.358479 16912 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5226ad7169074565b377e7f7621a0915 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:16:00.358583 16912 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5226ad7169074565b377e7f7621a0915 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:16:00.359645 16912 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5226ad7169074565b377e7f7621a0915 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5226ad7169074565b377e7f7621a0915" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42605 } }
I20260806 21:16:00.360044 16912 leader_election.cc:302] T 00000000000000000000000000000000 P 5226ad7169074565b377e7f7621a0915 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5226ad7169074565b377e7f7621a0915; no voters:
I20260806 21:16:00.360452 16912 leader_election.cc:288] T 00000000000000000000000000000000 P 5226ad7169074565b377e7f7621a0915 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:16:00.360572 16925 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5226ad7169074565b377e7f7621a0915 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:16:00.360813 16925 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5226ad7169074565b377e7f7621a0915 [term 1 LEADER]: Becoming Leader. State: Replica: 5226ad7169074565b377e7f7621a0915, State: Running, Role: LEADER
I20260806 21:16:00.361223 16925 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5226ad7169074565b377e7f7621a0915 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5226ad7169074565b377e7f7621a0915" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42605 } }
I20260806 21:16:00.361860 16912 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5226ad7169074565b377e7f7621a0915 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:16:00.363165 16927 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5226ad7169074565b377e7f7621a0915 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5226ad7169074565b377e7f7621a0915. Latest consensus state: current_term: 1 leader_uuid: "5226ad7169074565b377e7f7621a0915" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5226ad7169074565b377e7f7621a0915" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42605 } } }
I20260806 21:16:00.363340 16927 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5226ad7169074565b377e7f7621a0915 [sys.catalog]: This master's current role is: LEADER
I20260806 21:16:00.363937 16942 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:16:00.363183 16926 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5226ad7169074565b377e7f7621a0915 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5226ad7169074565b377e7f7621a0915" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5226ad7169074565b377e7f7621a0915" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42605 } } }
I20260806 21:16:00.364907 16926 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5226ad7169074565b377e7f7621a0915 [sys.catalog]: This master's current role is: LEADER
I20260806 21:16:00.367102 16942 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:16:00.373292 16942 catalog_manager.cc:1347] Generated new cluster ID: 4e0078713c1f4ae6b4f209584b786389
I20260806 21:16:00.373394 16942 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:16:00.394428 16942 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:16:00.394784 16942 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:16:00.408300 16942 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5226ad7169074565b377e7f7621a0915: Generated new TSK 0
I20260806 21:16:00.409099 16942 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:16:00.419874 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 16807
2026-08-06T21:16:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1571 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050686797304-17096-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:16:00 dist-test-slave-lgr5 krb5kdc[16956](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:16:00 dist-test-slave-lgr5 krb5kdc[16956](info): set up 2 sockets
Aug 06 21:16:00 dist-test-slave-lgr5 krb5kdc[16956](info): commencing operation
krb5kdc: starting...
W20260806 21:16:01.594543 17096 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Aug 06 21:16:01 dist-test-slave-lgr5 krb5kdc[16956](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050961, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:16:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:16:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050686797304-17096-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050686797304-17096-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 21:16:01.722640 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33349
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050686797304-17096-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33349
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:16:01.840510 16979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:16:01.840775 16979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:16:01.840821 16979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:16:01.844828 16979 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:16:01.844897 16979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:16:01.844918 16979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:16:01.844939 16979 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:16:01.844973 16979 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:16:01.850430 16979 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33349
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050686797304-17096-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16979
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:16:01.851490 16979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:16:01.852581 16979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:16:01.861373 16988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:16:01.861524 16987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:16:01.861824 16979 server_base.cc:1034] running on GCE node
W20260806 21:16:01.861961 16990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:16:01.862550 16979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:16:01.863731 16979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:16:01.864928 16979 hybrid_clock.cc:648] HybridClock initialized: now 1786050961864908 us; error 29 us; skew 500 ppm
Aug 06 21:16:01 dist-test-slave-lgr5 krb5kdc[16956](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050961, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:16:01.873584 16979 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 21:16:01.874699 16979 webserver.cc:460] Webserver started at http://127.0.0.1:38871/ using document root <none> and password file <none>
I20260806 21:16:01.875303 16979 fs_manager.cc:359] Metadata directory not provided
I20260806 21:16:01.875377 16979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:16:01.875581 16979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:16:01.877795 16979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "7d96c2bac2294a62a161130c17a12b04"
format_stamp: "Formatted at 2026-08-06 21:16:01 on dist-test-slave-lgr5"
server_key: "cecf77641f27c99c0ee38209cb4ce853"
server_key_iv: "97e9a69d2bfe3fafae1a82c77598feaa"
server_key_version: "encryptionkey@0"
I20260806 21:16:01.878360 16979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "7d96c2bac2294a62a161130c17a12b04"
format_stamp: "Formatted at 2026-08-06 21:16:01 on dist-test-slave-lgr5"
server_key: "cecf77641f27c99c0ee38209cb4ce853"
server_key_iv: "97e9a69d2bfe3fafae1a82c77598feaa"
server_key_version: "encryptionkey@0"
I20260806 21:16:01.882167 16979 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:16:01.884595 17000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:16:01.885838 16979 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 21:16:01.885982 16979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "7d96c2bac2294a62a161130c17a12b04"
format_stamp: "Formatted at 2026-08-06 21:16:01 on dist-test-slave-lgr5"
server_key: "cecf77641f27c99c0ee38209cb4ce853"
server_key_iv: "97e9a69d2bfe3fafae1a82c77598feaa"
server_key_version: "encryptionkey@0"
I20260806 21:16:01.886080 16979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:16:01.903539 16979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:16:01.906941 16979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:16:01.907143 16979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:16:01.914304 16979 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33349
I20260806 21:16:01.914318 17081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33349 every 8 connection(s)
I20260806 21:16:01.915302 16979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:16:01.918056 17082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:16:01.920310 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 16979
I20260806 21:16:01.920449 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:16:01.920774 17096 external_mini_cluster.cc:1428] Setting key e4e55d4e350de3b624c9a823e166c279
I20260806 21:16:01.924315 17082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d96c2bac2294a62a161130c17a12b04: Bootstrap starting.
I20260806 21:16:01.926828 17082 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d96c2bac2294a62a161130c17a12b04: Neither blocks nor log segments found. Creating new log.
I20260806 21:16:01.927605 17082 log.cc:824] T 00000000000000000000000000000000 P 7d96c2bac2294a62a161130c17a12b04: Log is configured to *not* fsync() on all Append() calls
Aug 06 21:16:01 dist-test-slave-lgr5 krb5kdc[16956](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050961, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 21:16:01.929894 17082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d96c2bac2294a62a161130c17a12b04: No bootstrap required, opened a new log
I20260806 21:16:01.933841 17082 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d96c2bac2294a62a161130c17a12b04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d96c2bac2294a62a161130c17a12b04" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33349 } }
I20260806 21:16:01.934137 17082 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d96c2bac2294a62a161130c17a12b04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:16:01.934198 17082 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d96c2bac2294a62a161130c17a12b04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d96c2bac2294a62a161130c17a12b04, State: Initialized, Role: FOLLOWER
I20260806 21:16:01.934644 17082 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d96c2bac2294a62a161130c17a12b04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d96c2bac2294a62a161130c17a12b04" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33349 } }
I20260806 21:16:01.934782 17082 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d96c2bac2294a62a161130c17a12b04 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:16:01.934846 17082 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d96c2bac2294a62a161130c17a12b04 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:16:01.934942 17088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:01.922127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.158:40109 (local address 127.0.0.1:33349)
0806 21:16:01.922528 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:01.922545 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:01.922584 (+ 39us) server_negotiation.cc:407] Connection header received
0806 21:16:01.923244 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:01.923261 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:01.923599 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:01.923880 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:01.924788 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:01.925795 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:01.926950 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:01.927443 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:01.927567 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:01.930353 (+ 2786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:01.930360 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:01.930376 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:16:01.930405 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:01.931973 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:01.932539 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:01.932544 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:01.932549 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:01.932660 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:01.932964 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:01.932967 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:01.932968 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:01.933364 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:01.933540 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:01.933854 (+ 314us) server_negotiation.cc:299] Negotiation successful
0806 21:16:01.934086 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":107,"threads_started":1}
I20260806 21:16:01.934944 17082 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d96c2bac2294a62a161130c17a12b04 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:16:01.935961 17082 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d96c2bac2294a62a161130c17a12b04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d96c2bac2294a62a161130c17a12b04" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33349 } }
I20260806 21:16:01.936324 17082 leader_election.cc:302] T 00000000000000000000000000000000 P 7d96c2bac2294a62a161130c17a12b04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d96c2bac2294a62a161130c17a12b04; no voters:
I20260806 21:16:01.936625 17082 leader_election.cc:288] T 00000000000000000000000000000000 P 7d96c2bac2294a62a161130c17a12b04 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:16:01.936748 17092 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d96c2bac2294a62a161130c17a12b04 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:16:01.936978 17092 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d96c2bac2294a62a161130c17a12b04 [term 1 LEADER]: Becoming Leader. State: Replica: 7d96c2bac2294a62a161130c17a12b04, State: Running, Role: LEADER
I20260806 21:16:01.937317 17092 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d96c2bac2294a62a161130c17a12b04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d96c2bac2294a62a161130c17a12b04" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33349 } }
I20260806 21:16:01.937870 17096 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:16:01.937973 17082 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d96c2bac2294a62a161130c17a12b04 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:16:01.938525 17092 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d96c2bac2294a62a161130c17a12b04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d96c2bac2294a62a161130c17a12b04. Latest consensus state: current_term: 1 leader_uuid: "7d96c2bac2294a62a161130c17a12b04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d96c2bac2294a62a161130c17a12b04" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33349 } } }
I20260806 21:16:01.938632 17092 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d96c2bac2294a62a161130c17a12b04 [sys.catalog]: This master's current role is: LEADER
I20260806 21:16:01.939016 17102 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:16:01.938964 17093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d96c2bac2294a62a161130c17a12b04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d96c2bac2294a62a161130c17a12b04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d96c2bac2294a62a161130c17a12b04" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33349 } } }
I20260806 21:16:01.939087 17093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d96c2bac2294a62a161130c17a12b04 [sys.catalog]: This master's current role is: LEADER
I20260806 21:16:01.942294 17102 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:16:01.949098 17088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:01.940227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.158:43441 (local address 127.0.0.1:33349)
0806 21:16:01.940323 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:01.940327 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:01.940879 (+ 552us) server_negotiation.cc:407] Connection header received
0806 21:16:01.941073 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:01.941076 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:01.941127 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:01.941177 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:01.942102 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:01.942675 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:01.943688 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:01.943957 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:01.944046 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:01.945440 (+ 1394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:01.945444 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:01.945446 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:16:01.945470 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:01.946966 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:01.947582 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:01.947586 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:01.947588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:01.947682 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:01.948032 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:01.948035 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:01.948037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:01.948312 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:01.948460 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:01.948819 (+ 359us) server_negotiation.cc:299] Negotiation successful
0806 21:16:01.948939 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 21:16:01.949852 17102 catalog_manager.cc:1347] Generated new cluster ID: 96c26363f1784a288ed3a9a070be3554
I20260806 21:16:01.949918 17102 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:16:01.980731 17102 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:16:01.982122 17102 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:16:01.991540 17102 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d96c2bac2294a62a161130c17a12b04: Generated new TSK 0
I20260806 21:16:01.992479 17102 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:16:02.007556 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 16979
W20260806 21:16:02.026791 16953 connection.cc:535] client connection to 127.0.0.1:33349 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33349): Connection reset by peer (error 104)
2026-08-06T21:16:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1574 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050686797304-17096-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:16:02 dist-test-slave-lgr5 krb5kdc[17131](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:16:02 dist-test-slave-lgr5 krb5kdc[17131](info): set up 2 sockets
Aug 06 21:16:02 dist-test-slave-lgr5 krb5kdc[17131](info): commencing operation
krb5kdc: starting...
W20260806 21:16:03.159636 17096 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Aug 06 21:16:03 dist-test-slave-lgr5 krb5kdc[17131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050963, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:16:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:16:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050686797304-17096-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050686797304-17096-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 21:16:03.291088 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38435
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050686797304-17096-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38435
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:16:03.406430 17160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:16:03.406715 17160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:16:03.406755 17160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:16:03.410718 17160 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:16:03.410784 17160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:16:03.410802 17160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:16:03.410818 17160 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:16:03.410846 17160 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:16:03.416270 17160 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38435
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050686797304-17096-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17160
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:16:03.417429 17160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:16:03.418481 17160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:16:03.427153 17166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:16:03.427151 17173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:16:03.427245 17168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:16:03.427328 17160 server_base.cc:1034] running on GCE node
I20260806 21:16:03.428059 17160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:16:03.429265 17160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:16:03.430418 17160 hybrid_clock.cc:648] HybridClock initialized: now 1786050963430397 us; error 37 us; skew 500 ppm
Aug 06 21:16:03 dist-test-slave-lgr5 krb5kdc[17131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050963, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:16:03.433207 17160 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 21:16:03.434144 17160 webserver.cc:460] Webserver started at http://127.0.0.1:44995/ using document root <none> and password file <none>
I20260806 21:16:03.434696 17160 fs_manager.cc:359] Metadata directory not provided
I20260806 21:16:03.434763 17160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:16:03.434969 17160 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:16:03.437184 17160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "96e8477fbe0e456db63220a3392c789a"
format_stamp: "Formatted at 2026-08-06 21:16:03 on dist-test-slave-lgr5"
server_key: "a370fd103467d2dceb9d81db0dd5e078"
server_key_iv: "f4a6cc612ba87bde65f56fd34652205b"
server_key_version: "encryptionkey@0"
I20260806 21:16:03.437768 17160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "96e8477fbe0e456db63220a3392c789a"
format_stamp: "Formatted at 2026-08-06 21:16:03 on dist-test-slave-lgr5"
server_key: "a370fd103467d2dceb9d81db0dd5e078"
server_key_iv: "f4a6cc612ba87bde65f56fd34652205b"
server_key_version: "encryptionkey@0"
I20260806 21:16:03.442027 17160 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:16:03.444747 17183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:16:03.446187 17160 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:16:03.446336 17160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "96e8477fbe0e456db63220a3392c789a"
format_stamp: "Formatted at 2026-08-06 21:16:03 on dist-test-slave-lgr5"
server_key: "a370fd103467d2dceb9d81db0dd5e078"
server_key_iv: "f4a6cc612ba87bde65f56fd34652205b"
server_key_version: "encryptionkey@0"
I20260806 21:16:03.446460 17160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:16:03.462226 17160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:16:03.465608 17160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:16:03.465811 17160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:16:03.473006 17160 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38435
I20260806 21:16:03.473034 17264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38435 every 8 connection(s)
I20260806 21:16:03.474013 17160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:16:03.477240 17267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:16:03.478924 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 17160
I20260806 21:16:03.479071 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:16:03.479409 17096 external_mini_cluster.cc:1428] Setting key 895ad73a1e4df8f6c1b7abf127ffca52
I20260806 21:16:03.484269 17267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96e8477fbe0e456db63220a3392c789a: Bootstrap starting.
I20260806 21:16:03.487107 17267 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96e8477fbe0e456db63220a3392c789a: Neither blocks nor log segments found. Creating new log.
I20260806 21:16:03.487908 17267 log.cc:824] T 00000000000000000000000000000000 P 96e8477fbe0e456db63220a3392c789a: Log is configured to *not* fsync() on all Append() calls
I20260806 21:16:03.489228 17270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:03.481031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.158:34965 (local address 127.0.0.1:38435)
0806 21:16:03.481495 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:03.481513 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:03.481551 (+ 38us) server_negotiation.cc:407] Connection header received
0806 21:16:03.482213 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:03.482231 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:03.482609 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:03.482903 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:03.483366 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:03.484443 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:03.485649 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:03.486140 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:03.486260 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:03.487302 (+ 1042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:03.487309 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:03.487324 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:16:03.487754 (+ 430us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:03.487822 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:16:03.487850 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:03.487973 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:03.488247 (+ 274us) server_negotiation.cc:299] Negotiation successful
0806 21:16:03.488421 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":126,"threads_started":1}
I20260806 21:16:03.490036 17267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96e8477fbe0e456db63220a3392c789a: No bootstrap required, opened a new log
I20260806 21:16:03.492552 17096 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:16:03.494647 17267 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96e8477fbe0e456db63220a3392c789a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96e8477fbe0e456db63220a3392c789a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38435 } }
I20260806 21:16:03.494999 17267 raft_consensus.cc:374] T 00000000000000000000000000000000 P 96e8477fbe0e456db63220a3392c789a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:16:03.495124 17267 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96e8477fbe0e456db63220a3392c789a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96e8477fbe0e456db63220a3392c789a, State: Initialized, Role: FOLLOWER
I20260806 21:16:03.495643 17267 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96e8477fbe0e456db63220a3392c789a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96e8477fbe0e456db63220a3392c789a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38435 } }
I20260806 21:16:03.495787 17267 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96e8477fbe0e456db63220a3392c789a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:16:03.495900 17267 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96e8477fbe0e456db63220a3392c789a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:16:03.496030 17267 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96e8477fbe0e456db63220a3392c789a [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:16:03.497267 17267 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96e8477fbe0e456db63220a3392c789a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96e8477fbe0e456db63220a3392c789a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38435 } }
I20260806 21:16:03.497714 17267 leader_election.cc:302] T 00000000000000000000000000000000 P 96e8477fbe0e456db63220a3392c789a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96e8477fbe0e456db63220a3392c789a; no voters:
I20260806 21:16:03.498114 17267 leader_election.cc:288] T 00000000000000000000000000000000 P 96e8477fbe0e456db63220a3392c789a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:16:03.498250 17280 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96e8477fbe0e456db63220a3392c789a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:16:03.498499 17280 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96e8477fbe0e456db63220a3392c789a [term 1 LEADER]: Becoming Leader. State: Replica: 96e8477fbe0e456db63220a3392c789a, State: Running, Role: LEADER
I20260806 21:16:03.498857 17280 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96e8477fbe0e456db63220a3392c789a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96e8477fbe0e456db63220a3392c789a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38435 } }
I20260806 21:16:03.499595 17267 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96e8477fbe0e456db63220a3392c789a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:16:03.499861 17270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:03.495090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.158:36899 (local address 127.0.0.1:38435)
0806 21:16:03.495244 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:03.495250 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:03.495551 (+ 301us) server_negotiation.cc:407] Connection header received
0806 21:16:03.495683 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:03.495686 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:03.495738 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:03.495817 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:03.496258 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:03.496793 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:03.497950 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:03.498266 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:03.498384 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:03.499355 (+ 971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:03.499359 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:03.499362 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:16:03.499377 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:03.499417 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:16:03.499431 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:03.499524 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:03.499678 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 21:16:03.499719 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 21:16:03.500463 17280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96e8477fbe0e456db63220a3392c789a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "96e8477fbe0e456db63220a3392c789a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96e8477fbe0e456db63220a3392c789a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38435 } } }
I20260806 21:16:03.500590 17280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96e8477fbe0e456db63220a3392c789a [sys.catalog]: This master's current role is: LEADER
I20260806 21:16:03.500793 17280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96e8477fbe0e456db63220a3392c789a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96e8477fbe0e456db63220a3392c789a. Latest consensus state: current_term: 1 leader_uuid: "96e8477fbe0e456db63220a3392c789a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96e8477fbe0e456db63220a3392c789a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38435 } } }
I20260806 21:16:03.500880 17280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96e8477fbe0e456db63220a3392c789a [sys.catalog]: This master's current role is: LEADER
I20260806 21:16:03.501297 17302 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:16:03.504642 17302 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:16:03.510656 17302 catalog_manager.cc:1347] Generated new cluster ID: 5a2502e863274156b9d9eac4279fb75c
I20260806 21:16:03.510735 17302 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:16:03.518606 17302 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:16:03.519719 17302 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:16:03.527823 17302 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 96e8477fbe0e456db63220a3392c789a: Generated new TSK 0
I20260806 21:16:03.528859 17302 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:16:03.541247 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 17160
W20260806 21:16:03.554374 17129 connection.cc:535] client connection to 127.0.0.1:38435 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38435): Connection reset by peer (error 104)
2026-08-06T21:16:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1527 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050686797304-17096-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:16:03 dist-test-slave-lgr5 krb5kdc[17316](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:16:03 dist-test-slave-lgr5 krb5kdc[17316](info): set up 2 sockets
Aug 06 21:16:03 dist-test-slave-lgr5 krb5kdc[17316](info): commencing operation
krb5kdc: starting...
W20260806 21:16:04.704591 17096 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Aug 06 21:16:04 dist-test-slave-lgr5 krb5kdc[17316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050964, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:16:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:16:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050686797304-17096-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050686797304-17096-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 21:16:04.839545 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37639
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050686797304-17096-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37639
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:16:04.956545 17330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:16:04.956862 17330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:16:04.956926 17330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:16:04.961179 17330 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:16:04.961272 17330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:16:04.961299 17330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:16:04.961320 17330 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:16:04.961342 17330 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:16:04.967020 17330 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37639
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050686797304-17096-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:16:04.968259 17330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:16:04.969329 17330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 21:16:04.979259 17330 server_base.cc:1034] running on GCE node
W20260806 21:16:04.979156 17337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:16:04.979153 17342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:16:04.979156 17339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:16:04.980262 17330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:16:04.981576 17330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:16:04.982776 17330 hybrid_clock.cc:648] HybridClock initialized: now 1786050964982754 us; error 40 us; skew 500 ppm
Aug 06 21:16:04 dist-test-slave-lgr5 krb5kdc[17316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050964, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:16:04.986187 17330 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 21:16:04.987478 17330 webserver.cc:460] Webserver started at http://127.0.0.1:46075/ using document root <none> and password file <none>
I20260806 21:16:04.988235 17330 fs_manager.cc:359] Metadata directory not provided
I20260806 21:16:04.988324 17330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:16:04.988569 17330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:16:04.990932 17330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "6c90a1da5dbe4326bc041661b7407e9e"
format_stamp: "Formatted at 2026-08-06 21:16:04 on dist-test-slave-lgr5"
server_key: "0553bd1d7f86dea57c1b4ad75ee873f8"
server_key_iv: "234cb3af8780b7b91dca1d5924ec6919"
server_key_version: "encryptionkey@0"
I20260806 21:16:04.991564 17330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "6c90a1da5dbe4326bc041661b7407e9e"
format_stamp: "Formatted at 2026-08-06 21:16:04 on dist-test-slave-lgr5"
server_key: "0553bd1d7f86dea57c1b4ad75ee873f8"
server_key_iv: "234cb3af8780b7b91dca1d5924ec6919"
server_key_version: "encryptionkey@0"
I20260806 21:16:04.995668 17330 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260806 21:16:04.998234 17353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:16:04.999511 17330 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 21:16:04.999626 17330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "6c90a1da5dbe4326bc041661b7407e9e"
format_stamp: "Formatted at 2026-08-06 21:16:04 on dist-test-slave-lgr5"
server_key: "0553bd1d7f86dea57c1b4ad75ee873f8"
server_key_iv: "234cb3af8780b7b91dca1d5924ec6919"
server_key_version: "encryptionkey@0"
I20260806 21:16:04.999738 17330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:16:05.004467 17330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:16:05.007680 17330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:16:05.007869 17330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:16:05.015554 17330 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37639
I20260806 21:16:05.015578 17438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37639 every 8 connection(s)
I20260806 21:16:05.016606 17330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:16:05.017131 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 17330
I20260806 21:16:05.017284 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:16:05.017572 17096 external_mini_cluster.cc:1428] Setting key 2f79973755acf48f563160fd74c259d2
I20260806 21:16:05.019744 17439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:16:05.022766 17444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:05.019005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.158:49883 (local address 127.0.0.1:37639)
0806 21:16:05.019408 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:05.019423 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:05.019459 (+ 36us) server_negotiation.cc:407] Connection header received
0806 21:16:05.020151 (+ 692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:05.020169 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:05.020448 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:05.020733 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:05.021567 (+ 834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:05.021572 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:05.021584 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:16:05.021594 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:05.021647 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:16:05.021665 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:05.021775 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:05.021882 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 21:16:05.022027 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":113,"threads_started":1}
I20260806 21:16:05.025660 17096 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:16:05.026355 17439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c90a1da5dbe4326bc041661b7407e9e: Bootstrap starting.
I20260806 21:16:05.029031 17439 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c90a1da5dbe4326bc041661b7407e9e: Neither blocks nor log segments found. Creating new log.
I20260806 21:16:05.029691 17444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:05.028036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.158:55271 (local address 127.0.0.1:37639)
0806 21:16:05.028182 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:05.028186 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:05.028422 (+ 236us) server_negotiation.cc:407] Connection header received
0806 21:16:05.028561 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:05.028564 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:05.028599 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:05.028662 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:05.029326 (+ 664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:05.029329 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:05.029332 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:16:05.029344 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:05.029370 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:16:05.029380 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:05.029449 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:05.029531 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 21:16:05.029573 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 21:16:05.029850 17439 log.cc:824] T 00000000000000000000000000000000 P 6c90a1da5dbe4326bc041661b7407e9e: Log is configured to *not* fsync() on all Append() calls
I20260806 21:16:05.031893 17439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c90a1da5dbe4326bc041661b7407e9e: No bootstrap required, opened a new log
I20260806 21:16:05.035678 17439 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c90a1da5dbe4326bc041661b7407e9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c90a1da5dbe4326bc041661b7407e9e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37639 } }
I20260806 21:16:05.035991 17439 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c90a1da5dbe4326bc041661b7407e9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:16:05.036062 17439 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c90a1da5dbe4326bc041661b7407e9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c90a1da5dbe4326bc041661b7407e9e, State: Initialized, Role: FOLLOWER
I20260806 21:16:05.036532 17439 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c90a1da5dbe4326bc041661b7407e9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c90a1da5dbe4326bc041661b7407e9e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37639 } }
I20260806 21:16:05.036672 17439 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c90a1da5dbe4326bc041661b7407e9e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:16:05.036741 17439 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c90a1da5dbe4326bc041661b7407e9e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:16:05.036839 17439 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c90a1da5dbe4326bc041661b7407e9e [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:16:05.037796 17439 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c90a1da5dbe4326bc041661b7407e9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c90a1da5dbe4326bc041661b7407e9e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37639 } }
I20260806 21:16:05.038138 17439 leader_election.cc:302] T 00000000000000000000000000000000 P 6c90a1da5dbe4326bc041661b7407e9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c90a1da5dbe4326bc041661b7407e9e; no voters:
I20260806 21:16:05.038473 17439 leader_election.cc:288] T 00000000000000000000000000000000 P 6c90a1da5dbe4326bc041661b7407e9e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:16:05.038594 17458 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c90a1da5dbe4326bc041661b7407e9e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:16:05.038823 17458 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c90a1da5dbe4326bc041661b7407e9e [term 1 LEADER]: Becoming Leader. State: Replica: 6c90a1da5dbe4326bc041661b7407e9e, State: Running, Role: LEADER
I20260806 21:16:05.039153 17458 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c90a1da5dbe4326bc041661b7407e9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c90a1da5dbe4326bc041661b7407e9e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37639 } }
I20260806 21:16:05.039824 17439 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c90a1da5dbe4326bc041661b7407e9e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:16:05.041038 17461 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c90a1da5dbe4326bc041661b7407e9e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c90a1da5dbe4326bc041661b7407e9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c90a1da5dbe4326bc041661b7407e9e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37639 } } }
I20260806 21:16:05.041040 17462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c90a1da5dbe4326bc041661b7407e9e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c90a1da5dbe4326bc041661b7407e9e. Latest consensus state: current_term: 1 leader_uuid: "6c90a1da5dbe4326bc041661b7407e9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c90a1da5dbe4326bc041661b7407e9e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37639 } } }
I20260806 21:16:05.041189 17462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c90a1da5dbe4326bc041661b7407e9e [sys.catalog]: This master's current role is: LEADER
I20260806 21:16:05.041188 17461 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c90a1da5dbe4326bc041661b7407e9e [sys.catalog]: This master's current role is: LEADER
I20260806 21:16:05.041656 17474 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:16:05.044382 17474 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:16:05.050038 17474 catalog_manager.cc:1347] Generated new cluster ID: fd314e1ee7e64ae6aef0f92f25233f84
I20260806 21:16:05.050119 17474 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:16:05.064800 17474 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:16:05.065186 17474 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:16:05.079087 17474 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c90a1da5dbe4326bc041661b7407e9e: Generated new TSK 0
I20260806 21:16:05.080037 17474 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:16:05.087692 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 17330
2026-08-06T21:16:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1547 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050686797304-17096-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:16:05 dist-test-slave-lgr5 krb5kdc[17491](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:16:05 dist-test-slave-lgr5 krb5kdc[17491](info): set up 2 sockets
Aug 06 21:16:05 dist-test-slave-lgr5 krb5kdc[17491](info): commencing operation
krb5kdc: starting...
W20260806 21:16:06.243772 17096 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Aug 06 21:16:06 dist-test-slave-lgr5 krb5kdc[17491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050966, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:16:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:16:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050686797304-17096-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050686797304-17096-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 21:16:06.371001 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34605
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050686797304-17096-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34605
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:16:06.487783 17507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:16:06.488142 17507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:16:06.488215 17507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:16:06.492282 17507 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:16:06.492360 17507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:16:06.492395 17507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:16:06.492419 17507 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:16:06.492445 17507 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:16:06.498123 17507 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34605
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050686797304-17096-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17507
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:16:06.499336 17507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:16:06.500483 17507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:16:06.509446 17512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:16:06.509454 17513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:16:06.509505 17507 server_base.cc:1034] running on GCE node
W20260806 21:16:06.509446 17515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:16:06.510382 17507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:16:06.511559 17507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:16:06.512749 17507 hybrid_clock.cc:648] HybridClock initialized: now 1786050966512737 us; error 41 us; skew 500 ppm
Aug 06 21:16:06 dist-test-slave-lgr5 krb5kdc[17491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050966, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:16:06.515822 17507 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 21:16:06.516927 17507 webserver.cc:460] Webserver started at http://127.0.0.1:35131/ using document root <none> and password file <none>
I20260806 21:16:06.517578 17507 fs_manager.cc:359] Metadata directory not provided
I20260806 21:16:06.517663 17507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:16:06.517889 17507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:16:06.520361 17507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "cb6bf2a659f44a77919026b894ba61ee"
format_stamp: "Formatted at 2026-08-06 21:16:06 on dist-test-slave-lgr5"
server_key: "b8d6c868f9eb6fd1b2f5b10fcbe549fa"
server_key_iv: "02394ed8c2d944adcec9bcad949741e4"
server_key_version: "encryptionkey@0"
I20260806 21:16:06.521085 17507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "cb6bf2a659f44a77919026b894ba61ee"
format_stamp: "Formatted at 2026-08-06 21:16:06 on dist-test-slave-lgr5"
server_key: "b8d6c868f9eb6fd1b2f5b10fcbe549fa"
server_key_iv: "02394ed8c2d944adcec9bcad949741e4"
server_key_version: "encryptionkey@0"
I20260806 21:16:06.525447 17507 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260806 21:16:06.528359 17525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:16:06.529724 17507 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:16:06.529879 17507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "cb6bf2a659f44a77919026b894ba61ee"
format_stamp: "Formatted at 2026-08-06 21:16:06 on dist-test-slave-lgr5"
server_key: "b8d6c868f9eb6fd1b2f5b10fcbe549fa"
server_key_iv: "02394ed8c2d944adcec9bcad949741e4"
server_key_version: "encryptionkey@0"
I20260806 21:16:06.530006 17507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:16:06.540802 17507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:16:06.544004 17507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:16:06.544217 17507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:16:06.551151 17507 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34605
I20260806 21:16:06.551168 17607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34605 every 8 connection(s)
I20260806 21:16:06.552145 17507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:16:06.554980 17608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:16:06.558609 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 17507
I20260806 21:16:06.558758 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:16:06.559038 17096 external_mini_cluster.cc:1428] Setting key 92fce242d3c145fb98df9b25e1cf63d0
I20260806 21:16:06.561138 17608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb6bf2a659f44a77919026b894ba61ee: Bootstrap starting.
I20260806 21:16:06.563923 17608 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb6bf2a659f44a77919026b894ba61ee: Neither blocks nor log segments found. Creating new log.
I20260806 21:16:06.564878 17608 log.cc:824] T 00000000000000000000000000000000 P cb6bf2a659f44a77919026b894ba61ee: Log is configured to *not* fsync() on all Append() calls
I20260806 21:16:06.567346 17608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb6bf2a659f44a77919026b894ba61ee: No bootstrap required, opened a new log
Aug 06 21:16:06 dist-test-slave-lgr5 krb5kdc[17491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050966, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 21:16:06.571867 17608 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb6bf2a659f44a77919026b894ba61ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb6bf2a659f44a77919026b894ba61ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34605 } }
I20260806 21:16:06.572190 17608 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb6bf2a659f44a77919026b894ba61ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:16:06.572259 17608 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb6bf2a659f44a77919026b894ba61ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb6bf2a659f44a77919026b894ba61ee, State: Initialized, Role: FOLLOWER
I20260806 21:16:06.572754 17608 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb6bf2a659f44a77919026b894ba61ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb6bf2a659f44a77919026b894ba61ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34605 } }
I20260806 21:16:06.572938 17608 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb6bf2a659f44a77919026b894ba61ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:16:06.573011 17608 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb6bf2a659f44a77919026b894ba61ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:16:06.573105 17608 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb6bf2a659f44a77919026b894ba61ee [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:16:06.574391 17608 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb6bf2a659f44a77919026b894ba61ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb6bf2a659f44a77919026b894ba61ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34605 } }
I20260806 21:16:06.574738 17608 leader_election.cc:302] T 00000000000000000000000000000000 P cb6bf2a659f44a77919026b894ba61ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb6bf2a659f44a77919026b894ba61ee; no voters:
I20260806 21:16:06.575062 17608 leader_election.cc:288] T 00000000000000000000000000000000 P cb6bf2a659f44a77919026b894ba61ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:16:06.575201 17615 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb6bf2a659f44a77919026b894ba61ee [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:16:06.575316 17613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:06.560551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.158:48051 (local address 127.0.0.1:34605)
0806 21:16:06.560994 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:06.561013 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:06.561057 (+ 44us) server_negotiation.cc:407] Connection header received
0806 21:16:06.561882 (+ 825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:06.561906 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:06.562347 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:06.562728 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:06.563455 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:06.564981 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:06.566033 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:06.566736 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:06.566906 (+ 170us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:06.569665 (+ 2759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:06.569672 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:06.569690 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:16:06.569725 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:06.571693 (+ 1968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:06.572259 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:06.572266 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:06.572272 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:06.572426 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:06.572772 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:06.572776 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:06.572777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:06.573332 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:06.573553 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:06.574080 (+ 527us) server_negotiation.cc:299] Negotiation successful
0806 21:16:06.574351 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":109,"threads_started":1}
I20260806 21:16:06.575464 17615 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb6bf2a659f44a77919026b894ba61ee [term 1 LEADER]: Becoming Leader. State: Replica: cb6bf2a659f44a77919026b894ba61ee, State: Running, Role: LEADER
I20260806 21:16:06.575896 17615 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb6bf2a659f44a77919026b894ba61ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb6bf2a659f44a77919026b894ba61ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34605 } }
I20260806 21:16:06.576406 17608 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb6bf2a659f44a77919026b894ba61ee [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:16:06.577605 17620 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb6bf2a659f44a77919026b894ba61ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb6bf2a659f44a77919026b894ba61ee. Latest consensus state: current_term: 1 leader_uuid: "cb6bf2a659f44a77919026b894ba61ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb6bf2a659f44a77919026b894ba61ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34605 } } }
I20260806 21:16:06.577992 17620 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb6bf2a659f44a77919026b894ba61ee [sys.catalog]: This master's current role is: LEADER
I20260806 21:16:06.577885 17619 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb6bf2a659f44a77919026b894ba61ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb6bf2a659f44a77919026b894ba61ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb6bf2a659f44a77919026b894ba61ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34605 } } }
I20260806 21:16:06.578316 17619 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb6bf2a659f44a77919026b894ba61ee [sys.catalog]: This master's current role is: LEADER
I20260806 21:16:06.579167 17640 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:16:06.579192 17096 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:16:06.582250 17640 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:16:06.588835 17640 catalog_manager.cc:1347] Generated new cluster ID: 12884916e9e64e10bea071766c31e1df
I20260806 21:16:06.588985 17640 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:16:06.589684 17613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:06.581221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.158:33109 (local address 127.0.0.1:34605)
0806 21:16:06.581326 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:06.581330 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:06.581588 (+ 258us) server_negotiation.cc:407] Connection header received
0806 21:16:06.581702 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:06.581704 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:06.581756 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:06.581841 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:06.582655 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:06.583447 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:06.584451 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:06.584692 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:06.584782 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:06.586100 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:06.586103 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:06.586105 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:16:06.586130 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:06.587631 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:06.588118 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:06.588122 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:06.588124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:06.588200 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:06.588553 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:06.588556 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:06.588558 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:06.588811 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:06.588940 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:06.589408 (+ 468us) server_negotiation.cc:299] Negotiation successful
0806 21:16:06.589523 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 21:16:06.637647 17640 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:16:06.638605 17640 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:16:06.645798 17640 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb6bf2a659f44a77919026b894ba61ee: Generated new TSK 0
I20260806 21:16:06.646554 17640 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:16:06.660262 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 17507
W20260806 21:16:06.675637 17489 connection.cc:535] client connection to 127.0.0.1:34605 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34605): Connection reset by peer (error 104)
2026-08-06T21:16:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1573 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050686797304-17096-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:16:06 dist-test-slave-lgr5 krb5kdc[17657](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:16:06 dist-test-slave-lgr5 krb5kdc[17657](info): set up 2 sockets
Aug 06 21:16:06 dist-test-slave-lgr5 krb5kdc[17657](info): commencing operation
krb5kdc: starting...
W20260806 21:16:07.811990 17096 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Aug 06 21:16:07 dist-test-slave-lgr5 krb5kdc[17657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050967, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:16:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:16:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050686797304-17096-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050686797304-17096-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 21:16:07.941201 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38703
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050686797304-17096-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38703
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:16:08.063582 17678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:16:08.063910 17678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:16:08.063982 17678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:16:08.068188 17678 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:16:08.068271 17678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:16:08.068298 17678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:16:08.068321 17678 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:16:08.068351 17678 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:16:08.074278 17678 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38703
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050686797304-17096-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17678
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:16:08.075575 17678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:16:08.076781 17678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:16:08.085817 17684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:16:08.085791 17689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:16:08.085786 17685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:16:08.085855 17678 server_base.cc:1034] running on GCE node
I20260806 21:16:08.086750 17678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:16:08.087970 17678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:16:08.089150 17678 hybrid_clock.cc:648] HybridClock initialized: now 1786050968089135 us; error 32 us; skew 500 ppm
Aug 06 21:16:08 dist-test-slave-lgr5 krb5kdc[17657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050968, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:16:08.096323 17678 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 21:16:08.097438 17678 webserver.cc:460] Webserver started at http://127.0.0.1:44593/ using document root <none> and password file <none>
I20260806 21:16:08.098040 17678 fs_manager.cc:359] Metadata directory not provided
I20260806 21:16:08.098107 17678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:16:08.098318 17678 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:16:08.100595 17678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "e2162852e76c429ba952159a9f8236f3"
format_stamp: "Formatted at 2026-08-06 21:16:08 on dist-test-slave-lgr5"
server_key: "5d2fee61975dd191fef96653b8b6baf6"
server_key_iv: "146c6ba5e18f9287dd4c4efcb146f003"
server_key_version: "encryptionkey@0"
I20260806 21:16:08.101189 17678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "e2162852e76c429ba952159a9f8236f3"
format_stamp: "Formatted at 2026-08-06 21:16:08 on dist-test-slave-lgr5"
server_key: "5d2fee61975dd191fef96653b8b6baf6"
server_key_iv: "146c6ba5e18f9287dd4c4efcb146f003"
server_key_version: "encryptionkey@0"
I20260806 21:16:08.105129 17678 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:16:08.107638 17699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:16:08.109004 17678 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:16:08.109123 17678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "e2162852e76c429ba952159a9f8236f3"
format_stamp: "Formatted at 2026-08-06 21:16:08 on dist-test-slave-lgr5"
server_key: "5d2fee61975dd191fef96653b8b6baf6"
server_key_iv: "146c6ba5e18f9287dd4c4efcb146f003"
server_key_version: "encryptionkey@0"
I20260806 21:16:08.109233 17678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:16:08.122843 17678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:16:08.126204 17678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:16:08.126411 17678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:16:08.134014 17678 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38703
I20260806 21:16:08.134034 17788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38703 every 8 connection(s)
I20260806 21:16:08.135051 17678 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:16:08.138073 17790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:16:08.139569 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 17678
I20260806 21:16:08.139694 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:16:08.139976 17096 external_mini_cluster.cc:1428] Setting key 7705c44bbd77fbbbd4d34c79929c90dc
I20260806 21:16:08.144994 17790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2162852e76c429ba952159a9f8236f3: Bootstrap starting.
I20260806 21:16:08.147715 17790 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e2162852e76c429ba952159a9f8236f3: Neither blocks nor log segments found. Creating new log.
I20260806 21:16:08.148623 17790 log.cc:824] T 00000000000000000000000000000000 P e2162852e76c429ba952159a9f8236f3: Log is configured to *not* fsync() on all Append() calls
I20260806 21:16:08.150184 17794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:08.141448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.158:58179 (local address 127.0.0.1:38703)
0806 21:16:08.141900 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:08.141920 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:08.141965 (+ 45us) server_negotiation.cc:407] Connection header received
0806 21:16:08.142793 (+ 828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:08.142819 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:08.143264 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:08.143656 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:08.144116 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:08.145526 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:08.146477 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:08.147158 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:08.147333 (+ 175us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:08.148189 (+ 856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:08.148195 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:08.148212 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:16:08.148722 (+ 510us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:08.148795 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:16:08.148828 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:08.148959 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:08.149126 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 21:16:08.149266 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":113,"threads_started":1}
I20260806 21:16:08.150875 17790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2162852e76c429ba952159a9f8236f3: No bootstrap required, opened a new log
I20260806 21:16:08.153064 17096 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:16:08.154767 17790 raft_consensus.cc:348] T 00000000000000000000000000000000 P e2162852e76c429ba952159a9f8236f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2162852e76c429ba952159a9f8236f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38703 } }
I20260806 21:16:08.155121 17790 raft_consensus.cc:374] T 00000000000000000000000000000000 P e2162852e76c429ba952159a9f8236f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:16:08.155198 17790 raft_consensus.cc:729] T 00000000000000000000000000000000 P e2162852e76c429ba952159a9f8236f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2162852e76c429ba952159a9f8236f3, State: Initialized, Role: FOLLOWER
I20260806 21:16:08.155700 17790 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2162852e76c429ba952159a9f8236f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2162852e76c429ba952159a9f8236f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38703 } }
I20260806 21:16:08.155898 17790 raft_consensus.cc:388] T 00000000000000000000000000000000 P e2162852e76c429ba952159a9f8236f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:16:08.155956 17802 raft_consensus.cc:482] T 00000000000000000000000000000000 P e2162852e76c429ba952159a9f8236f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 21:16:08.156121 17802 raft_consensus.cc:504] T 00000000000000000000000000000000 P e2162852e76c429ba952159a9f8236f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2162852e76c429ba952159a9f8236f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38703 } }
I20260806 21:16:08.156454 17790 raft_consensus.cc:482] T 00000000000000000000000000000000 P e2162852e76c429ba952159a9f8236f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:16:08.156471 17802 leader_election.cc:302] T 00000000000000000000000000000000 P e2162852e76c429ba952159a9f8236f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e2162852e76c429ba952159a9f8236f3; no voters:
I20260806 21:16:08.156541 17790 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e2162852e76c429ba952159a9f8236f3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:16:08.156859 17802 leader_election.cc:288] T 00000000000000000000000000000000 P e2162852e76c429ba952159a9f8236f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20260806 21:16:08.157657 17790 raft_consensus.cc:504] T 00000000000000000000000000000000 P e2162852e76c429ba952159a9f8236f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2162852e76c429ba952159a9f8236f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38703 } }
I20260806 21:16:08.157779 17790 leader_election.cc:302] T 00000000000000000000000000000000 P e2162852e76c429ba952159a9f8236f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e2162852e76c429ba952159a9f8236f3; no voters:
I20260806 21:16:08.157869 17790 leader_election.cc:288] T 00000000000000000000000000000000 P e2162852e76c429ba952159a9f8236f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:16:08.157903 17802 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e2162852e76c429ba952159a9f8236f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:16:08.158094 17802 raft_consensus.cc:686] T 00000000000000000000000000000000 P e2162852e76c429ba952159a9f8236f3 [term 1 LEADER]: Becoming Leader. State: Replica: e2162852e76c429ba952159a9f8236f3, State: Running, Role: LEADER
I20260806 21:16:08.158481 17802 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2162852e76c429ba952159a9f8236f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2162852e76c429ba952159a9f8236f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38703 } }
I20260806 21:16:08.159179 17790 sys_catalog.cc:564] T 00000000000000000000000000000000 P e2162852e76c429ba952159a9f8236f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:16:08.159375 17794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:08.155086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.158:38671 (local address 127.0.0.1:38703)
0806 21:16:08.155223 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:08.155227 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:08.155486 (+ 259us) server_negotiation.cc:407] Connection header received
0806 21:16:08.155608 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:08.155611 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:08.155662 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:08.155745 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:08.156036 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:08.156714 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:08.157582 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:08.157833 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:08.157925 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:08.158880 (+ 955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:08.158884 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:08.158887 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:16:08.158901 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:08.158934 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:16:08.158945 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:08.159012 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:08.159197 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 21:16:08.159235 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 21:16:08.159765 17802 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2162852e76c429ba952159a9f8236f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2162852e76c429ba952159a9f8236f3. Latest consensus state: current_term: 1 leader_uuid: "e2162852e76c429ba952159a9f8236f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2162852e76c429ba952159a9f8236f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38703 } } }
I20260806 21:16:08.159890 17802 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2162852e76c429ba952159a9f8236f3 [sys.catalog]: This master's current role is: LEADER
I20260806 21:16:08.161226 17803 raft_consensus.cc:2741] T 00000000000000000000000000000000 P e2162852e76c429ba952159a9f8236f3 [term 1 LEADER]: Leader pre-election decision vote started in defunct term 0: won
I20260806 21:16:08.161638 17819 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:16:08.162671 17804 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2162852e76c429ba952159a9f8236f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e2162852e76c429ba952159a9f8236f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2162852e76c429ba952159a9f8236f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38703 } } }
I20260806 21:16:08.162766 17804 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2162852e76c429ba952159a9f8236f3 [sys.catalog]: This master's current role is: LEADER
I20260806 21:16:08.164623 17819 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:16:08.170498 17819 catalog_manager.cc:1347] Generated new cluster ID: 6672c585f5834cb4a4301845adc4b894
I20260806 21:16:08.170595 17819 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:16:08.179464 17819 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:16:08.180531 17819 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:16:08.188165 17819 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e2162852e76c429ba952159a9f8236f3: Generated new TSK 0
I20260806 21:16:08.189253 17819 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:16:08.198498 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 17678
W20260806 21:16:08.210793 17655 connection.cc:535] client connection to 127.0.0.1:38703 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38703): Connection reset by peer (error 104)
2026-08-06T21:16:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1535 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050686797304-17096-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:16:08 dist-test-slave-lgr5 krb5kdc[17835](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:16:08 dist-test-slave-lgr5 krb5kdc[17835](info): set up 2 sockets
Aug 06 21:16:08 dist-test-slave-lgr5 krb5kdc[17835](info): commencing operation
krb5kdc: starting...
W20260806 21:16:09.354693 17096 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Aug 06 21:16:09 dist-test-slave-lgr5 krb5kdc[17835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:16:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:16:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050686797304-17096-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050686797304-17096-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 21:16:09.487012 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36193
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050686797304-17096-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36193
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:16:09.601696 17855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:16:09.601987 17855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:16:09.602029 17855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:16:09.606058 17855 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:16:09.606118 17855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:16:09.606137 17855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:16:09.606153 17855 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:16:09.606166 17855 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:16:09.611423 17855 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36193
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050686797304-17096-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17855
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:16:09.612541 17855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:16:09.613554 17855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:16:09.621816 17861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:16:09.621953 17855 server_base.cc:1034] running on GCE node
W20260806 21:16:09.621847 17860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:16:09.621886 17864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:16:09.622596 17855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:16:09.623667 17855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:16:09.624832 17855 hybrid_clock.cc:648] HybridClock initialized: now 1786050969624804 us; error 42 us; skew 500 ppm
Aug 06 21:16:09 dist-test-slave-lgr5 krb5kdc[17835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050969, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:16:09.627491 17855 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 21:16:09.628405 17855 webserver.cc:460] Webserver started at http://127.0.0.1:43493/ using document root <none> and password file <none>
I20260806 21:16:09.628947 17855 fs_manager.cc:359] Metadata directory not provided
I20260806 21:16:09.629035 17855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:16:09.629251 17855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:16:09.631281 17855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "85f6f9edd63c4e209071308e68fa7889"
format_stamp: "Formatted at 2026-08-06 21:16:09 on dist-test-slave-lgr5"
server_key: "530c08e6ef002b2b157e95cf509e6dac"
server_key_iv: "be41df14d24793d2ae829c192faf674c"
server_key_version: "encryptionkey@0"
I20260806 21:16:09.631836 17855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "85f6f9edd63c4e209071308e68fa7889"
format_stamp: "Formatted at 2026-08-06 21:16:09 on dist-test-slave-lgr5"
server_key: "530c08e6ef002b2b157e95cf509e6dac"
server_key_iv: "be41df14d24793d2ae829c192faf674c"
server_key_version: "encryptionkey@0"
I20260806 21:16:09.635540 17855 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:16:09.637930 17877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:16:09.639150 17855 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 21:16:09.639282 17855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "85f6f9edd63c4e209071308e68fa7889"
format_stamp: "Formatted at 2026-08-06 21:16:09 on dist-test-slave-lgr5"
server_key: "530c08e6ef002b2b157e95cf509e6dac"
server_key_iv: "be41df14d24793d2ae829c192faf674c"
server_key_version: "encryptionkey@0"
I20260806 21:16:09.639397 17855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:16:09.644127 17855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:16:09.647374 17855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:16:09.647567 17855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:16:09.655334 17855 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36193
I20260806 21:16:09.655339 17968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36193 every 8 connection(s)
I20260806 21:16:09.656441 17855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:16:09.659353 17969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:16:09.664625 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 17855
I20260806 21:16:09.664769 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:16:09.665051 17096 external_mini_cluster.cc:1428] Setting key 792622ccc52a01013f54bfe57ab44786
I20260806 21:16:09.665135 17969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85f6f9edd63c4e209071308e68fa7889: Bootstrap starting.
I20260806 21:16:09.667696 17969 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 85f6f9edd63c4e209071308e68fa7889: Neither blocks nor log segments found. Creating new log.
I20260806 21:16:09.668572 17969 log.cc:824] T 00000000000000000000000000000000 P 85f6f9edd63c4e209071308e68fa7889: Log is configured to *not* fsync() on all Append() calls
I20260806 21:16:09.670503 17972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:09.666421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.158:56777 (local address 127.0.0.1:36193)
0806 21:16:09.666907 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:09.666923 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:09.666991 (+ 68us) server_negotiation.cc:407] Connection header received
0806 21:16:09.667648 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:09.667665 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:09.667998 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:09.668347 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:09.669213 (+ 866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:09.669219 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:09.669236 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:16:09.669251 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:09.669328 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:16:09.669361 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:09.669520 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:09.669607 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 21:16:09.669784 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":137,"threads_started":1}
I20260806 21:16:09.670760 17969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85f6f9edd63c4e209071308e68fa7889: No bootstrap required, opened a new log
I20260806 21:16:09.673406 17096 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:16:09.675203 17969 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85f6f9edd63c4e209071308e68fa7889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85f6f9edd63c4e209071308e68fa7889" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36193 } }
I20260806 21:16:09.675496 17969 raft_consensus.cc:374] T 00000000000000000000000000000000 P 85f6f9edd63c4e209071308e68fa7889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:16:09.675558 17969 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85f6f9edd63c4e209071308e68fa7889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85f6f9edd63c4e209071308e68fa7889, State: Initialized, Role: FOLLOWER
I20260806 21:16:09.676105 17969 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85f6f9edd63c4e209071308e68fa7889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85f6f9edd63c4e209071308e68fa7889" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36193 } }
I20260806 21:16:09.676278 17969 raft_consensus.cc:388] T 00000000000000000000000000000000 P 85f6f9edd63c4e209071308e68fa7889 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:16:09.676348 17969 raft_consensus.cc:482] T 00000000000000000000000000000000 P 85f6f9edd63c4e209071308e68fa7889 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:16:09.676453 17969 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85f6f9edd63c4e209071308e68fa7889 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:16:09.677440 17972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:09.675712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.158:41579 (local address 127.0.0.1:36193)
0806 21:16:09.675851 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:09.675855 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:09.676111 (+ 256us) server_negotiation.cc:407] Connection header received
0806 21:16:09.676271 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:09.676275 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:09.676315 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:09.676419 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:09.677082 (+ 663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:09.677085 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:09.677087 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:16:09.677098 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:09.677123 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:16:09.677133 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:09.677199 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:09.677284 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 21:16:09.677317 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 21:16:09.677685 17969 raft_consensus.cc:504] T 00000000000000000000000000000000 P 85f6f9edd63c4e209071308e68fa7889 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85f6f9edd63c4e209071308e68fa7889" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36193 } }
I20260806 21:16:09.678061 17969 leader_election.cc:302] T 00000000000000000000000000000000 P 85f6f9edd63c4e209071308e68fa7889 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 85f6f9edd63c4e209071308e68fa7889; no voters:
I20260806 21:16:09.678426 17969 leader_election.cc:288] T 00000000000000000000000000000000 P 85f6f9edd63c4e209071308e68fa7889 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:16:09.678524 17981 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 85f6f9edd63c4e209071308e68fa7889 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:16:09.678738 17981 raft_consensus.cc:686] T 00000000000000000000000000000000 P 85f6f9edd63c4e209071308e68fa7889 [term 1 LEADER]: Becoming Leader. State: Replica: 85f6f9edd63c4e209071308e68fa7889, State: Running, Role: LEADER
I20260806 21:16:09.679092 17981 consensus_queue.cc:237] T 00000000000000000000000000000000 P 85f6f9edd63c4e209071308e68fa7889 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85f6f9edd63c4e209071308e68fa7889" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36193 } }
I20260806 21:16:09.679724 17969 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85f6f9edd63c4e209071308e68fa7889 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:16:09.681041 17982 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85f6f9edd63c4e209071308e68fa7889 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "85f6f9edd63c4e209071308e68fa7889" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85f6f9edd63c4e209071308e68fa7889" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36193 } } }
I20260806 21:16:09.681061 17983 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85f6f9edd63c4e209071308e68fa7889 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85f6f9edd63c4e209071308e68fa7889. Latest consensus state: current_term: 1 leader_uuid: "85f6f9edd63c4e209071308e68fa7889" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85f6f9edd63c4e209071308e68fa7889" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36193 } } }
I20260806 21:16:09.681205 17982 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85f6f9edd63c4e209071308e68fa7889 [sys.catalog]: This master's current role is: LEADER
I20260806 21:16:09.681227 17983 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85f6f9edd63c4e209071308e68fa7889 [sys.catalog]: This master's current role is: LEADER
I20260806 21:16:09.681608 17999 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:16:09.685174 17999 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:16:09.690826 17999 catalog_manager.cc:1347] Generated new cluster ID: 0665b93574244ce19f0d629d5bc8215b
I20260806 21:16:09.690912 17999 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:16:09.724239 17999 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:16:09.724653 17999 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:16:09.730777 17999 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 85f6f9edd63c4e209071308e68fa7889: Generated new TSK 0
I20260806 21:16:09.731741 17999 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:16:09.743204 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 17855
2026-08-06T21:16:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1543 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050686797304-17096-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:16:09 dist-test-slave-lgr5 krb5kdc[18018](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:16:09 dist-test-slave-lgr5 krb5kdc[18018](info): set up 2 sockets
Aug 06 21:16:09 dist-test-slave-lgr5 krb5kdc[18018](info): commencing operation
krb5kdc: starting...
W20260806 21:16:10.891410 17096 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Aug 06 21:16:10 dist-test-slave-lgr5 krb5kdc[18018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050970, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:16:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:16:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.178.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
I20260806 21:16:11.023769 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:46573
--webserver_interface=127.16.178.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:44005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:46573
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:16:11.142232 18037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:16:11.142555 18037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:16:11.142599 18037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:16:11.146741 18037 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:16:11.146816 18037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:16:11.146834 18037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:16:11.146849 18037 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:16:11.146867 18037 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:16:11.152427 18037 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:44005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:46573
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:46573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.178.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18037
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:16:11.153509 18037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:16:11.154505 18037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:16:11.163116 18050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:16:11.163128 18046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:16:11.163131 18047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:16:11.163388 18037 server_base.cc:1034] running on GCE node
I20260806 21:16:11.163980 18037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:16:11.165138 18037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:16:11.166327 18037 hybrid_clock.cc:648] HybridClock initialized: now 1786050971166311 us; error 26 us; skew 500 ppm
Aug 06 21:16:11 dist-test-slave-lgr5 krb5kdc[18018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050971, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:16:11.169163 18037 init.cc:377] Logged in from keytab as kudu/127.16.178.62@KRBTEST.COM (short username kudu)
I20260806 21:16:11.170065 18037 webserver.cc:460] Webserver started at http://127.16.178.62:46319/ using document root <none> and password file <none>
I20260806 21:16:11.170612 18037 fs_manager.cc:359] Metadata directory not provided
I20260806 21:16:11.170681 18037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:16:11.170888 18037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:16:11.172991 18037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "d566119918bb40c68ba03993cd32e981"
format_stamp: "Formatted at 2026-08-06 21:16:11 on dist-test-slave-lgr5"
server_key: "391acd5c2347400ee844c09aded9d01d"
server_key_iv: "004efd7a6256c19a8bdc4ccb7ebe78f2"
server_key_version: "encryptionkey@0"
I20260806 21:16:11.173539 18037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "d566119918bb40c68ba03993cd32e981"
format_stamp: "Formatted at 2026-08-06 21:16:11 on dist-test-slave-lgr5"
server_key: "391acd5c2347400ee844c09aded9d01d"
server_key_iv: "004efd7a6256c19a8bdc4ccb7ebe78f2"
server_key_version: "encryptionkey@0"
I20260806 21:16:11.177184 18037 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260806 21:16:11.179489 18058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:16:11.180709 18037 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 21:16:11.180825 18037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "d566119918bb40c68ba03993cd32e981"
format_stamp: "Formatted at 2026-08-06 21:16:11 on dist-test-slave-lgr5"
server_key: "391acd5c2347400ee844c09aded9d01d"
server_key_iv: "004efd7a6256c19a8bdc4ccb7ebe78f2"
server_key_version: "encryptionkey@0"
I20260806 21:16:11.180924 18037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:16:11.199847 18037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:16:11.203085 18037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:16:11.203296 18037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:16:11.210222 18037 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.62:46573
I20260806 21:16:11.210235 18143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.62:46573 every 8 connection(s)
I20260806 21:16:11.211225 18037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:16:11.214080 18144 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:16:11.219856 18144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d566119918bb40c68ba03993cd32e981: Bootstrap starting.
I20260806 21:16:11.221453 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 18037
I20260806 21:16:11.221593 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:16:11.221864 17096 external_mini_cluster.cc:1428] Setting key 1330e776096d6a24c26eeab0f4f3fa37
I20260806 21:16:11.222677 18144 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d566119918bb40c68ba03993cd32e981: Neither blocks nor log segments found. Creating new log.
I20260806 21:16:11.223726 18144 log.cc:824] T 00000000000000000000000000000000 P d566119918bb40c68ba03993cd32e981: Log is configured to *not* fsync() on all Append() calls
I20260806 21:16:11.225865 18144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d566119918bb40c68ba03993cd32e981: No bootstrap required, opened a new log
I20260806 21:16:11.229523 18144 raft_consensus.cc:348] T 00000000000000000000000000000000 P d566119918bb40c68ba03993cd32e981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d566119918bb40c68ba03993cd32e981" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 46573 } }
I20260806 21:16:11.229789 18144 raft_consensus.cc:374] T 00000000000000000000000000000000 P d566119918bb40c68ba03993cd32e981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:16:11.229859 18144 raft_consensus.cc:729] T 00000000000000000000000000000000 P d566119918bb40c68ba03993cd32e981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d566119918bb40c68ba03993cd32e981, State: Initialized, Role: FOLLOWER
I20260806 21:16:11.230327 18144 consensus_queue.cc:260] T 00000000000000000000000000000000 P d566119918bb40c68ba03993cd32e981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d566119918bb40c68ba03993cd32e981" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 46573 } }
I20260806 21:16:11.230470 18144 raft_consensus.cc:388] T 00000000000000000000000000000000 P d566119918bb40c68ba03993cd32e981 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
Aug 06 21:16:11 dist-test-slave-lgr5 krb5kdc[18018](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050970, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:16:11.230541 18144 raft_consensus.cc:482] T 00000000000000000000000000000000 P d566119918bb40c68ba03993cd32e981 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:16:11.230643 18144 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d566119918bb40c68ba03993cd32e981 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:16:11.231796 18144 raft_consensus.cc:504] T 00000000000000000000000000000000 P d566119918bb40c68ba03993cd32e981 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d566119918bb40c68ba03993cd32e981" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 46573 } }
I20260806 21:16:11.232208 18144 leader_election.cc:302] T 00000000000000000000000000000000 P d566119918bb40c68ba03993cd32e981 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d566119918bb40c68ba03993cd32e981; no voters:
I20260806 21:16:11.232614 18144 leader_election.cc:288] T 00000000000000000000000000000000 P d566119918bb40c68ba03993cd32e981 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:16:11.232708 18152 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d566119918bb40c68ba03993cd32e981 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:16:11.232929 18152 raft_consensus.cc:686] T 00000000000000000000000000000000 P d566119918bb40c68ba03993cd32e981 [term 1 LEADER]: Becoming Leader. State: Replica: d566119918bb40c68ba03993cd32e981, State: Running, Role: LEADER
I20260806 21:16:11.233304 18152 consensus_queue.cc:237] T 00000000000000000000000000000000 P d566119918bb40c68ba03993cd32e981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d566119918bb40c68ba03993cd32e981" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 46573 } }
I20260806 21:16:11.234022 18144 sys_catalog.cc:564] T 00000000000000000000000000000000 P d566119918bb40c68ba03993cd32e981 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:16:11.235087 18153 sys_catalog.cc:455] T 00000000000000000000000000000000 P d566119918bb40c68ba03993cd32e981 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d566119918bb40c68ba03993cd32e981" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d566119918bb40c68ba03993cd32e981" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 46573 } } }
I20260806 21:16:11.235216 18153 sys_catalog.cc:458] T 00000000000000000000000000000000 P d566119918bb40c68ba03993cd32e981 [sys.catalog]: This master's current role is: LEADER
I20260806 21:16:11.235618 18154 sys_catalog.cc:455] T 00000000000000000000000000000000 P d566119918bb40c68ba03993cd32e981 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d566119918bb40c68ba03993cd32e981. Latest consensus state: current_term: 1 leader_uuid: "d566119918bb40c68ba03993cd32e981" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d566119918bb40c68ba03993cd32e981" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 46573 } } }
I20260806 21:16:11.235782 18154 sys_catalog.cc:458] T 00000000000000000000000000000000 P d566119918bb40c68ba03993cd32e981 [sys.catalog]: This master's current role is: LEADER
I20260806 21:16:11.237262 18149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:11.223101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51019 (local address 127.16.178.62:46573)
0806 21:16:11.223626 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:11.223644 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:11.223687 (+ 43us) server_negotiation.cc:407] Connection header received
0806 21:16:11.224464 (+ 777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:11.224494 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:11.224856 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:11.225186 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:11.226000 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:11.227013 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:11.227875 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:11.228432 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:11.228623 (+ 191us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:11.231331 (+ 2708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:11.231337 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:11.231354 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:16:11.231388 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:11.233452 (+ 2064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:11.234084 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:11.234090 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:11.234097 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:11.234245 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:11.234670 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:11.234673 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:11.234675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:11.235283 (+ 608us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:11.235485 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:11.236064 (+ 579us) server_negotiation.cc:299] Negotiation successful
0806 21:16:11.236371 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":160,"threads_started":1}
W20260806 21:16:11.239137 18171 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d566119918bb40c68ba03993cd32e981: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 21:16:11.239215 18171 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 21:16:11.239275 18162 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:16:11.240074 18162 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:16:11.241080 17096 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:16:11.245617 18162 catalog_manager.cc:1347] Generated new cluster ID: 2a4b731b91054ed4b74f8a4e8bb0129b
I20260806 21:16:11.245677 18162 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:16:11.252902 18149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:11.243376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43697 (local address 127.16.178.62:46573)
0806 21:16:11.243490 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:11.243494 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:11.243722 (+ 228us) server_negotiation.cc:407] Connection header received
0806 21:16:11.243825 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:11.243829 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:11.243873 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:11.243935 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:11.244708 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:11.245534 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:11.246750 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:11.247130 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:11.247210 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:11.249076 (+ 1866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:11.249080 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:11.249083 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:16:11.249110 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:11.250756 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:11.251348 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:11.251352 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:11.251353 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:11.251438 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:11.251815 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:11.251819 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:11.251821 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:11.252047 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:11.252212 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:11.252629 (+ 417us) server_negotiation.cc:299] Negotiation successful
0806 21:16:11.252735 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 21:16:11.258652 18162 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:16:11.259896 18162 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:16:11.270097 18162 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d566119918bb40c68ba03993cd32e981: Generated new TSK 0
I20260806 21:16:11.271072 18162 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:16:11.284344 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 18037
W20260806 21:16:11.298882 18014 connection.cc:535] client connection to 127.16.178.62:46573 recv error: Network error: failed to read from TLS socket (remote: 127.16.178.62:46573): Connection reset by peer (error 104)
2026-08-06T21:16:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1544 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050686797304-17096-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:16:11 dist-test-slave-lgr5 krb5kdc[18186](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:16:11 dist-test-slave-lgr5 krb5kdc[18186](info): set up 2 sockets
Aug 06 21:16:11 dist-test-slave-lgr5 krb5kdc[18186](info): commencing operation
krb5kdc: starting...
W20260806 21:16:12.439821 17096 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Aug 06 21:16:12 dist-test-slave-lgr5 krb5kdc[18186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050972, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:16:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:16:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.178.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
I20260806 21:16:12.566821 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:41565
--webserver_interface=127.16.178.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:43437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:41565
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:16:12.679692 18207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:16:12.679996 18207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:16:12.680037 18207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:16:12.684163 18207 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:16:12.684255 18207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:16:12.684276 18207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:16:12.684293 18207 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:16:12.684314 18207 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:16:12.690183 18207 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:43437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:41565
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:41565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.178.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18207
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:16:12.691381 18207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:16:12.692467 18207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:16:12.700959 18217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:16:12.701021 18207 server_base.cc:1034] running on GCE node
W20260806 21:16:12.700959 18215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:16:12.700924 18219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:16:12.701777 18207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:16:12.702909 18207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:16:12.704118 18207 hybrid_clock.cc:648] HybridClock initialized: now 1786050972704086 us; error 39 us; skew 500 ppm
Aug 06 21:16:12 dist-test-slave-lgr5 krb5kdc[18186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050972, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:16:12.706854 18207 init.cc:377] Logged in from keytab as kudu/127.16.178.62@KRBTEST.COM (short username kudu)
I20260806 21:16:12.707913 18207 webserver.cc:460] Webserver started at http://127.16.178.62:38507/ using document root <none> and password file <none>
I20260806 21:16:12.708511 18207 fs_manager.cc:359] Metadata directory not provided
I20260806 21:16:12.708583 18207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:16:12.708792 18207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:16:12.710916 18207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "6b43d19f5ca942379546bf6e74312ae0"
format_stamp: "Formatted at 2026-08-06 21:16:12 on dist-test-slave-lgr5"
server_key: "052499e72ecc419b2a13fced49e56b0a"
server_key_iv: "012c67345382fddbe238f8b0b4b1505f"
server_key_version: "encryptionkey@0"
I20260806 21:16:12.711477 18207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "6b43d19f5ca942379546bf6e74312ae0"
format_stamp: "Formatted at 2026-08-06 21:16:12 on dist-test-slave-lgr5"
server_key: "052499e72ecc419b2a13fced49e56b0a"
server_key_iv: "012c67345382fddbe238f8b0b4b1505f"
server_key_version: "encryptionkey@0"
I20260806 21:16:12.715427 18207 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 21:16:12.718044 18228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:16:12.719368 18207 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 21:16:12.719507 18207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "6b43d19f5ca942379546bf6e74312ae0"
format_stamp: "Formatted at 2026-08-06 21:16:12 on dist-test-slave-lgr5"
server_key: "052499e72ecc419b2a13fced49e56b0a"
server_key_iv: "012c67345382fddbe238f8b0b4b1505f"
server_key_version: "encryptionkey@0"
I20260806 21:16:12.719621 18207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:16:12.735517 18207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:16:12.738394 18207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:16:12.738582 18207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:16:12.745880 18207 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.62:41565
I20260806 21:16:12.745910 18308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.62:41565 every 8 connection(s)
I20260806 21:16:12.746979 18207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:16:12.750041 18309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:16:12.754128 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 18207
I20260806 21:16:12.754283 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:16:12.754590 17096 external_mini_cluster.cc:1428] Setting key 2f0eb3cd04e66bb10039d6c763cf4120
I20260806 21:16:12.756400 18309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b43d19f5ca942379546bf6e74312ae0: Bootstrap starting.
I20260806 21:16:12.758826 18309 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b43d19f5ca942379546bf6e74312ae0: Neither blocks nor log segments found. Creating new log.
I20260806 21:16:12.759649 18309 log.cc:824] T 00000000000000000000000000000000 P 6b43d19f5ca942379546bf6e74312ae0: Log is configured to *not* fsync() on all Append() calls
I20260806 21:16:12.761924 18309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b43d19f5ca942379546bf6e74312ae0: No bootstrap required, opened a new log
I20260806 21:16:12.763916 18313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:12.756124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57171 (local address 127.16.178.62:41565)
0806 21:16:12.756574 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:12.756589 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:12.756628 (+ 39us) server_negotiation.cc:407] Connection header received
0806 21:16:12.757277 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:12.757296 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:12.757630 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:12.757912 (+ 282us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:12.758255 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:12.759297 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:12.760415 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:12.760959 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:12.761081 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:12.761973 (+ 892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:12.761979 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:12.761992 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:16:12.762405 (+ 413us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:12.762464 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:16:12.762489 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:12.762595 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:12.762780 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 21:16:12.762932 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":132,"threads_started":1}
I20260806 21:16:12.765149 18309 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b43d19f5ca942379546bf6e74312ae0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b43d19f5ca942379546bf6e74312ae0" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 41565 } }
I20260806 21:16:12.765424 18309 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b43d19f5ca942379546bf6e74312ae0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:16:12.765481 18309 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b43d19f5ca942379546bf6e74312ae0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b43d19f5ca942379546bf6e74312ae0, State: Initialized, Role: FOLLOWER
I20260806 21:16:12.765887 18309 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b43d19f5ca942379546bf6e74312ae0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b43d19f5ca942379546bf6e74312ae0" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 41565 } }
I20260806 21:16:12.766057 18309 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b43d19f5ca942379546bf6e74312ae0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:16:12.766125 18309 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b43d19f5ca942379546bf6e74312ae0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:16:12.766216 18309 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b43d19f5ca942379546bf6e74312ae0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:16:12.766930 17096 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:16:12.767388 18309 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b43d19f5ca942379546bf6e74312ae0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b43d19f5ca942379546bf6e74312ae0" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 41565 } }
I20260806 21:16:12.767766 18309 leader_election.cc:302] T 00000000000000000000000000000000 P 6b43d19f5ca942379546bf6e74312ae0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b43d19f5ca942379546bf6e74312ae0; no voters:
I20260806 21:16:12.768177 18309 leader_election.cc:288] T 00000000000000000000000000000000 P 6b43d19f5ca942379546bf6e74312ae0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:16:12.768841 18321 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b43d19f5ca942379546bf6e74312ae0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:16:12.769253 18321 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b43d19f5ca942379546bf6e74312ae0 [term 1 LEADER]: Becoming Leader. State: Replica: 6b43d19f5ca942379546bf6e74312ae0, State: Running, Role: LEADER
I20260806 21:16:12.769847 18309 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b43d19f5ca942379546bf6e74312ae0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:16:12.770632 18321 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b43d19f5ca942379546bf6e74312ae0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b43d19f5ca942379546bf6e74312ae0" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 41565 } }
I20260806 21:16:12.772643 18315 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b43d19f5ca942379546bf6e74312ae0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b43d19f5ca942379546bf6e74312ae0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b43d19f5ca942379546bf6e74312ae0" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 41565 } } }
I20260806 21:16:12.772782 18315 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b43d19f5ca942379546bf6e74312ae0 [sys.catalog]: This master's current role is: LEADER
I20260806 21:16:12.773046 18322 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b43d19f5ca942379546bf6e74312ae0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b43d19f5ca942379546bf6e74312ae0. Latest consensus state: current_term: 1 leader_uuid: "6b43d19f5ca942379546bf6e74312ae0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b43d19f5ca942379546bf6e74312ae0" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 41565 } } }
I20260806 21:16:12.773200 18322 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b43d19f5ca942379546bf6e74312ae0 [sys.catalog]: This master's current role is: LEADER
I20260806 21:16:12.774540 18313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:12.769212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43053 (local address 127.16.178.62:41565)
0806 21:16:12.769320 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:12.769325 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:12.769708 (+ 383us) server_negotiation.cc:407] Connection header received
0806 21:16:12.769894 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:12.769898 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:12.769949 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:12.770000 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:12.770426 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:12.771220 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:12.772372 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:12.772727 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:12.772825 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:12.773893 (+ 1068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:12.773897 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:12.773899 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:16:12.773915 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:12.773945 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:16:12.773958 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:12.774046 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:12.774376 (+ 330us) server_negotiation.cc:299] Negotiation successful
0806 21:16:12.774414 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260806 21:16:12.775197 18339 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6b43d19f5ca942379546bf6e74312ae0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 21:16:12.775277 18339 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 21:16:12.775353 18340 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:16:12.776065 18340 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:16:12.781222 18340 catalog_manager.cc:1347] Generated new cluster ID: b247cca13e5a4c91a66bc6bdc81e9a5e
I20260806 21:16:12.781299 18340 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:16:12.809258 18340 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:16:12.810619 18340 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:16:12.818753 18340 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b43d19f5ca942379546bf6e74312ae0: Generated new TSK 0
I20260806 21:16:12.819504 18340 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:16:12.828878 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 18207
W20260806 21:16:12.844729 18183 connection.cc:535] client connection to 127.16.178.62:41565 recv error: Network error: failed to read from TLS socket (remote: 127.16.178.62:41565): Connection reset by peer (error 104)
2026-08-06T21:16:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1546 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050686797304-17096-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:16:12 dist-test-slave-lgr5 krb5kdc[18347](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:16:12 dist-test-slave-lgr5 krb5kdc[18347](info): set up 2 sockets
Aug 06 21:16:12 dist-test-slave-lgr5 krb5kdc[18347](info): commencing operation
krb5kdc: starting...
W20260806 21:16:13.987072 17096 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Aug 06 21:16:14 dist-test-slave-lgr5 krb5kdc[18347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:16:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:16:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.178.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
I20260806 21:16:14.119014 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:43109
--webserver_interface=127.16.178.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:44957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:43109
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:16:14.235962 18368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:16:14.236305 18368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:16:14.236368 18368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:16:14.240456 18368 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:16:14.240535 18368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:16:14.240562 18368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:16:14.240584 18368 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:16:14.240607 18368 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:16:14.246203 18368 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:44957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:43109
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:43109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.178.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:16:14.247341 18368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:16:14.248427 18368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:16:14.257762 18376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:16:14.257762 18375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:16:14.257875 18368 server_base.cc:1034] running on GCE node
W20260806 21:16:14.257779 18378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:16:14.258744 18368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:16:14.260026 18368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:16:14.261224 18368 hybrid_clock.cc:648] HybridClock initialized: now 1786050974261210 us; error 30 us; skew 500 ppm
Aug 06 21:16:14 dist-test-slave-lgr5 krb5kdc[18347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050974, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:16:14.267663 18368 init.cc:377] Logged in from keytab as kudu/127.16.178.62@KRBTEST.COM (short username kudu)
I20260806 21:16:14.268699 18368 webserver.cc:460] Webserver started at http://127.16.178.62:44161/ using document root <none> and password file <none>
I20260806 21:16:14.269270 18368 fs_manager.cc:359] Metadata directory not provided
I20260806 21:16:14.269341 18368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:16:14.269555 18368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:16:14.271775 18368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "eb9edb0c5a254167bcb5182685c2717f"
format_stamp: "Formatted at 2026-08-06 21:16:14 on dist-test-slave-lgr5"
server_key: "aab4da3403315dbe0a8f04f8e9eb9a78"
server_key_iv: "b52aa600bdf102021c46c186550a3995"
server_key_version: "encryptionkey@0"
I20260806 21:16:14.272375 18368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "eb9edb0c5a254167bcb5182685c2717f"
format_stamp: "Formatted at 2026-08-06 21:16:14 on dist-test-slave-lgr5"
server_key: "aab4da3403315dbe0a8f04f8e9eb9a78"
server_key_iv: "b52aa600bdf102021c46c186550a3995"
server_key_version: "encryptionkey@0"
I20260806 21:16:14.276461 18368 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 21:16:14.278855 18389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:16:14.280128 18368 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 21:16:14.280267 18368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "eb9edb0c5a254167bcb5182685c2717f"
format_stamp: "Formatted at 2026-08-06 21:16:14 on dist-test-slave-lgr5"
server_key: "aab4da3403315dbe0a8f04f8e9eb9a78"
server_key_iv: "b52aa600bdf102021c46c186550a3995"
server_key_version: "encryptionkey@0"
I20260806 21:16:14.280371 18368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:16:14.284925 18368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:16:14.288035 18368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:16:14.288237 18368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:16:14.295152 18368 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.62:43109
I20260806 21:16:14.295192 18462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.62:43109 every 8 connection(s)
I20260806 21:16:14.296205 18368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:16:14.297564 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 18368
I20260806 21:16:14.297708 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:16:14.297989 17096 external_mini_cluster.cc:1428] Setting key 809ef01e291b779420a52ed2c3c1b052
I20260806 21:16:14.299356 18463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:16:14.303287 18466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:14.299283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51417 (local address 127.16.178.62:43109)
0806 21:16:14.299710 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:14.299726 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:14.299762 (+ 36us) server_negotiation.cc:407] Connection header received
0806 21:16:14.300456 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:14.300475 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:14.300776 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:14.301091 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:14.301997 (+ 906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:14.302002 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:14.302014 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:16:14.302024 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:14.302078 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:16:14.302099 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:14.302215 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:14.302322 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 21:16:14.302477 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":146,"threads_started":1}
I20260806 21:16:14.306272 18463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb9edb0c5a254167bcb5182685c2717f: Bootstrap starting.
I20260806 21:16:14.306479 17096 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:16:14.308961 18463 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb9edb0c5a254167bcb5182685c2717f: Neither blocks nor log segments found. Creating new log.
I20260806 21:16:14.309847 18463 log.cc:824] T 00000000000000000000000000000000 P eb9edb0c5a254167bcb5182685c2717f: Log is configured to *not* fsync() on all Append() calls
I20260806 21:16:14.310281 18466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:14.308623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47585 (local address 127.16.178.62:43109)
0806 21:16:14.308760 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:14.308764 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:14.308904 (+ 140us) server_negotiation.cc:407] Connection header received
0806 21:16:14.309023 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:14.309026 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:14.309071 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:14.309162 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:14.309856 (+ 694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:14.309860 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:14.309863 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:16:14.309875 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:14.309905 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:16:14.309919 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:14.310014 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:14.310111 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 21:16:14.310154 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 21:16:14.312247 18463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb9edb0c5a254167bcb5182685c2717f: No bootstrap required, opened a new log
I20260806 21:16:14.315990 18463 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb9edb0c5a254167bcb5182685c2717f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb9edb0c5a254167bcb5182685c2717f" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 43109 } }
I20260806 21:16:14.316303 18463 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb9edb0c5a254167bcb5182685c2717f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:16:14.316375 18463 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb9edb0c5a254167bcb5182685c2717f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb9edb0c5a254167bcb5182685c2717f, State: Initialized, Role: FOLLOWER
I20260806 21:16:14.316854 18463 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb9edb0c5a254167bcb5182685c2717f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb9edb0c5a254167bcb5182685c2717f" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 43109 } }
I20260806 21:16:14.317031 18463 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb9edb0c5a254167bcb5182685c2717f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:16:14.317103 18463 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb9edb0c5a254167bcb5182685c2717f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:16:14.317215 18463 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb9edb0c5a254167bcb5182685c2717f [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:16:14.318203 18463 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb9edb0c5a254167bcb5182685c2717f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb9edb0c5a254167bcb5182685c2717f" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 43109 } }
I20260806 21:16:14.318544 18463 leader_election.cc:302] T 00000000000000000000000000000000 P eb9edb0c5a254167bcb5182685c2717f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb9edb0c5a254167bcb5182685c2717f; no voters:
I20260806 21:16:14.318847 18463 leader_election.cc:288] T 00000000000000000000000000000000 P eb9edb0c5a254167bcb5182685c2717f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:16:14.318977 18474 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb9edb0c5a254167bcb5182685c2717f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:16:14.319209 18474 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb9edb0c5a254167bcb5182685c2717f [term 1 LEADER]: Becoming Leader. State: Replica: eb9edb0c5a254167bcb5182685c2717f, State: Running, Role: LEADER
I20260806 21:16:14.319550 18474 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb9edb0c5a254167bcb5182685c2717f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb9edb0c5a254167bcb5182685c2717f" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 43109 } }
I20260806 21:16:14.320099 18463 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb9edb0c5a254167bcb5182685c2717f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:16:14.321189 18475 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb9edb0c5a254167bcb5182685c2717f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb9edb0c5a254167bcb5182685c2717f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb9edb0c5a254167bcb5182685c2717f" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 43109 } } }
I20260806 21:16:14.321331 18475 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb9edb0c5a254167bcb5182685c2717f [sys.catalog]: This master's current role is: LEADER
I20260806 21:16:14.321548 18476 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb9edb0c5a254167bcb5182685c2717f [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb9edb0c5a254167bcb5182685c2717f. Latest consensus state: current_term: 1 leader_uuid: "eb9edb0c5a254167bcb5182685c2717f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb9edb0c5a254167bcb5182685c2717f" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 43109 } } }
I20260806 21:16:14.321650 18476 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb9edb0c5a254167bcb5182685c2717f [sys.catalog]: This master's current role is: LEADER
I20260806 21:16:14.321983 18488 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:16:14.325874 18488 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:16:14.332499 18488 catalog_manager.cc:1347] Generated new cluster ID: 4a6c64e4de0d4681b03033bda6816607
I20260806 21:16:14.332600 18488 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:16:14.343669 18488 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:16:14.344051 18488 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:16:14.353528 18488 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb9edb0c5a254167bcb5182685c2717f: Generated new TSK 0
I20260806 21:16:14.354317 18488 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:16:14.357939 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 18368
2026-08-06T21:16:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1528 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050686797304-17096-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:16:14 dist-test-slave-lgr5 krb5kdc[18502](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:16:14 dist-test-slave-lgr5 krb5kdc[18502](info): set up 2 sockets
Aug 06 21:16:14 dist-test-slave-lgr5 krb5kdc[18502](info): commencing operation
krb5kdc: starting...
W20260806 21:16:15.537662 17096 mini_kdc.cc:121] Time spent starting KDC: real 1.150s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Aug 06 21:16:15 dist-test-slave-lgr5 krb5kdc[18502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:16:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:16:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.178.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
I20260806 21:16:15.665040 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:35509
--webserver_interface=127.16.178.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:34343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:35509
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:16:15.781832 18521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:16:15.782131 18521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:16:15.782182 18521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:16:15.786361 18521 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:16:15.786437 18521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:16:15.786455 18521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:16:15.786471 18521 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:16:15.786484 18521 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:16:15.791989 18521 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:34343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:35509
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:35509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.178.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18521
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:16:15.793123 18521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:16:15.794140 18521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:16:15.802582 18534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:16:15.802594 18533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:16:15.802597 18538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:16:15.802644 18521 server_base.cc:1034] running on GCE node
I20260806 21:16:15.803404 18521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:16:15.804622 18521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:16:15.805787 18521 hybrid_clock.cc:648] HybridClock initialized: now 1786050975805774 us; error 30 us; skew 500 ppm
Aug 06 21:16:15 dist-test-slave-lgr5 krb5kdc[18502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050975, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:16:15.808686 18521 init.cc:377] Logged in from keytab as kudu/127.16.178.62@KRBTEST.COM (short username kudu)
I20260806 21:16:15.809643 18521 webserver.cc:460] Webserver started at http://127.16.178.62:38107/ using document root <none> and password file <none>
I20260806 21:16:15.810201 18521 fs_manager.cc:359] Metadata directory not provided
I20260806 21:16:15.810266 18521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:16:15.810454 18521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:16:15.812665 18521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "dcef8fc5c7fb4466a0b4bfc567485a08"
format_stamp: "Formatted at 2026-08-06 21:16:15 on dist-test-slave-lgr5"
server_key: "0c6106b2773904c7fc887f8e633bd167"
server_key_iv: "0d58a0f0b7e0bc68c6646536688d0759"
server_key_version: "encryptionkey@0"
I20260806 21:16:15.813234 18521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "dcef8fc5c7fb4466a0b4bfc567485a08"
format_stamp: "Formatted at 2026-08-06 21:16:15 on dist-test-slave-lgr5"
server_key: "0c6106b2773904c7fc887f8e633bd167"
server_key_iv: "0d58a0f0b7e0bc68c6646536688d0759"
server_key_version: "encryptionkey@0"
I20260806 21:16:15.817224 18521 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 21:16:15.819926 18548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:16:15.821405 18521 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 21:16:15.821553 18521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "dcef8fc5c7fb4466a0b4bfc567485a08"
format_stamp: "Formatted at 2026-08-06 21:16:15 on dist-test-slave-lgr5"
server_key: "0c6106b2773904c7fc887f8e633bd167"
server_key_iv: "0d58a0f0b7e0bc68c6646536688d0759"
server_key_version: "encryptionkey@0"
I20260806 21:16:15.821668 18521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:16:15.848815 18521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:16:15.851588 18521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:16:15.851745 18521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:16:15.858462 18521 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.62:35509
I20260806 21:16:15.858484 18621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.62:35509 every 8 connection(s)
I20260806 21:16:15.859404 18521 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:16:15.862159 18622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:16:15.863329 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 18521
I20260806 21:16:15.863451 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:16:15.863722 17096 external_mini_cluster.cc:1428] Setting key 264b2c985d132eedd6a255a44911fb4d
I20260806 21:16:15.868235 18622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcef8fc5c7fb4466a0b4bfc567485a08: Bootstrap starting.
I20260806 21:16:15.870811 18622 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dcef8fc5c7fb4466a0b4bfc567485a08: Neither blocks nor log segments found. Creating new log.
I20260806 21:16:15.871731 18622 log.cc:824] T 00000000000000000000000000000000 P dcef8fc5c7fb4466a0b4bfc567485a08: Log is configured to *not* fsync() on all Append() calls
Aug 06 21:16:15 dist-test-slave-lgr5 krb5kdc[18502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:16:15.873730 18622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcef8fc5c7fb4466a0b4bfc567485a08: No bootstrap required, opened a new log
I20260806 21:16:15.877373 18622 raft_consensus.cc:348] T 00000000000000000000000000000000 P dcef8fc5c7fb4466a0b4bfc567485a08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcef8fc5c7fb4466a0b4bfc567485a08" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 35509 } }
I20260806 21:16:15.877609 18622 raft_consensus.cc:374] T 00000000000000000000000000000000 P dcef8fc5c7fb4466a0b4bfc567485a08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:16:15.877678 18622 raft_consensus.cc:729] T 00000000000000000000000000000000 P dcef8fc5c7fb4466a0b4bfc567485a08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcef8fc5c7fb4466a0b4bfc567485a08, State: Initialized, Role: FOLLOWER
I20260806 21:16:15.877751 18629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:15.865112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33397 (local address 127.16.178.62:35509)
0806 21:16:15.865503 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:15.865519 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:15.865555 (+ 36us) server_negotiation.cc:407] Connection header received
0806 21:16:15.866172 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:15.866188 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:15.866513 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:15.866778 (+ 265us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:15.867704 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:15.868754 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:15.869924 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:15.870423 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:15.870555 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:15.873294 (+ 2739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:15.873299 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:15.873312 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:16:15.873338 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:15.874907 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:15.875330 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:15.875335 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:15.875340 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:15.875516 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:15.875776 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:15.875780 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:15.875781 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:15.876233 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:15.876428 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:15.876677 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 21:16:15.876971 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":98,"threads_started":1}
I20260806 21:16:15.878209 18622 consensus_queue.cc:260] T 00000000000000000000000000000000 P dcef8fc5c7fb4466a0b4bfc567485a08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcef8fc5c7fb4466a0b4bfc567485a08" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 35509 } }
I20260806 21:16:15.878377 18622 raft_consensus.cc:388] T 00000000000000000000000000000000 P dcef8fc5c7fb4466a0b4bfc567485a08 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:16:15.878443 18622 raft_consensus.cc:482] T 00000000000000000000000000000000 P dcef8fc5c7fb4466a0b4bfc567485a08 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:16:15.878545 18622 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dcef8fc5c7fb4466a0b4bfc567485a08 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:16:15.879455 18622 raft_consensus.cc:504] T 00000000000000000000000000000000 P dcef8fc5c7fb4466a0b4bfc567485a08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcef8fc5c7fb4466a0b4bfc567485a08" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 35509 } }
I20260806 21:16:15.879815 18622 leader_election.cc:302] T 00000000000000000000000000000000 P dcef8fc5c7fb4466a0b4bfc567485a08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dcef8fc5c7fb4466a0b4bfc567485a08; no voters:
I20260806 21:16:15.880149 18622 leader_election.cc:288] T 00000000000000000000000000000000 P dcef8fc5c7fb4466a0b4bfc567485a08 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:16:15.880311 18631 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dcef8fc5c7fb4466a0b4bfc567485a08 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:16:15.880622 18631 raft_consensus.cc:686] T 00000000000000000000000000000000 P dcef8fc5c7fb4466a0b4bfc567485a08 [term 1 LEADER]: Becoming Leader. State: Replica: dcef8fc5c7fb4466a0b4bfc567485a08, State: Running, Role: LEADER
I20260806 21:16:15.880805 17096 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:16:15.881405 18622 sys_catalog.cc:564] T 00000000000000000000000000000000 P dcef8fc5c7fb4466a0b4bfc567485a08 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:16:15.882788 18631 consensus_queue.cc:237] T 00000000000000000000000000000000 P dcef8fc5c7fb4466a0b4bfc567485a08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcef8fc5c7fb4466a0b4bfc567485a08" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 35509 } }
I20260806 21:16:15.884939 18635 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcef8fc5c7fb4466a0b4bfc567485a08 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dcef8fc5c7fb4466a0b4bfc567485a08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcef8fc5c7fb4466a0b4bfc567485a08" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 35509 } } }
I20260806 21:16:15.885438 18635 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcef8fc5c7fb4466a0b4bfc567485a08 [sys.catalog]: This master's current role is: LEADER
I20260806 21:16:15.884976 18636 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcef8fc5c7fb4466a0b4bfc567485a08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dcef8fc5c7fb4466a0b4bfc567485a08. Latest consensus state: current_term: 1 leader_uuid: "dcef8fc5c7fb4466a0b4bfc567485a08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcef8fc5c7fb4466a0b4bfc567485a08" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 35509 } } }
I20260806 21:16:15.886011 18636 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcef8fc5c7fb4466a0b4bfc567485a08 [sys.catalog]: This master's current role is: LEADER
W20260806 21:16:15.888399 18657 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dcef8fc5c7fb4466a0b4bfc567485a08: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 21:16:15.888514 18657 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 21:16:15.888620 18658 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:16:15.890182 18658 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:16:15.892019 18629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:15.883557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46349 (local address 127.16.178.62:35509)
0806 21:16:15.883677 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:15.883682 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:15.883848 (+ 166us) server_negotiation.cc:407] Connection header received
0806 21:16:15.884012 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:15.884016 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:15.884068 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:15.884184 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:15.885152 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:15.885885 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:15.887190 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:15.887535 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:15.887658 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:15.889187 (+ 1529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:15.889191 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:15.889194 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:16:15.889220 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:15.890488 (+ 1268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:15.890866 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:15.890870 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:15.890872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:15.890948 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:15.891196 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:15.891200 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:15.891202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:15.891379 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:15.891493 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:15.891737 (+ 244us) server_negotiation.cc:299] Negotiation successful
0806 21:16:15.891847 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 21:16:15.896232 18658 catalog_manager.cc:1347] Generated new cluster ID: 2747896e1e9e4b6e9a6e566bdea30679
I20260806 21:16:15.896353 18658 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:16:15.904719 18658 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:16:15.905995 18658 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:16:15.913590 18658 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dcef8fc5c7fb4466a0b4bfc567485a08: Generated new TSK 0
I20260806 21:16:15.914345 18658 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:16:15.925822 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 18521
2026-08-06T21:16:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1561 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050686797304-17096-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:16:15 dist-test-slave-lgr5 krb5kdc[18668](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:16:15 dist-test-slave-lgr5 krb5kdc[18668](info): set up 2 sockets
Aug 06 21:16:15 dist-test-slave-lgr5 krb5kdc[18668](info): commencing operation
krb5kdc: starting...
W20260806 21:16:17.077176 17096 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Aug 06 21:16:17 dist-test-slave-lgr5 krb5kdc[18668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:16:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:16:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.178.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
I20260806 21:16:17.211844 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:34403
--webserver_interface=127.16.178.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:39421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:34403
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:16:17.330708 18688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:16:17.331064 18688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:16:17.331135 18688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:16:17.335532 18688 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:16:17.335639 18688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:16:17.335671 18688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:16:17.335700 18688 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:16:17.335736 18688 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:16:17.341976 18688 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:39421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:34403
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:34403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.178.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:16:17.343207 18688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:16:17.344352 18688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:16:17.353606 18696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:16:17.353619 18697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:16:17.353744 18688 server_base.cc:1034] running on GCE node
W20260806 21:16:17.353609 18701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:16:17.354532 18688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:16:17.355718 18688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:16:17.356890 18688 hybrid_clock.cc:648] HybridClock initialized: now 1786050977356871 us; error 33 us; skew 500 ppm
Aug 06 21:16:17 dist-test-slave-lgr5 krb5kdc[18668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050977, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:16:17.359788 18688 init.cc:377] Logged in from keytab as kudu/127.16.178.62@KRBTEST.COM (short username kudu)
I20260806 21:16:17.360865 18688 webserver.cc:460] Webserver started at http://127.16.178.62:38489/ using document root <none> and password file <none>
I20260806 21:16:17.361507 18688 fs_manager.cc:359] Metadata directory not provided
I20260806 21:16:17.361588 18688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:16:17.361811 18688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:16:17.364192 18688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "e07ea765eb7d4e27b48d36029a6efaa2"
format_stamp: "Formatted at 2026-08-06 21:16:17 on dist-test-slave-lgr5"
server_key: "68c9412bde5749c61c244e89c26bd7b3"
server_key_iv: "a4b4020479eb7ee7cfa31559660e4b06"
server_key_version: "encryptionkey@0"
I20260806 21:16:17.364840 18688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "e07ea765eb7d4e27b48d36029a6efaa2"
format_stamp: "Formatted at 2026-08-06 21:16:17 on dist-test-slave-lgr5"
server_key: "68c9412bde5749c61c244e89c26bd7b3"
server_key_iv: "a4b4020479eb7ee7cfa31559660e4b06"
server_key_version: "encryptionkey@0"
I20260806 21:16:17.369081 18688 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 21:16:17.371658 18713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:16:17.372956 18688 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260806 21:16:17.373096 18688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "e07ea765eb7d4e27b48d36029a6efaa2"
format_stamp: "Formatted at 2026-08-06 21:16:17 on dist-test-slave-lgr5"
server_key: "68c9412bde5749c61c244e89c26bd7b3"
server_key_iv: "a4b4020479eb7ee7cfa31559660e4b06"
server_key_version: "encryptionkey@0"
I20260806 21:16:17.373211 18688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:16:17.405937 18688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:16:17.409534 18688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:16:17.409747 18688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:16:17.416697 18688 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.62:34403
I20260806 21:16:17.416728 18796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.62:34403 every 8 connection(s)
I20260806 21:16:17.417724 18688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:16:17.419973 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 18688
I20260806 21:16:17.420131 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:16:17.420416 17096 external_mini_cluster.cc:1428] Setting key 42e36b01f47d63ec360e64a3e841fd99
I20260806 21:16:17.420717 18800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:16:17.428211 18800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e07ea765eb7d4e27b48d36029a6efaa2: Bootstrap starting.
I20260806 21:16:17.430282 18803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:17.421917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39995 (local address 127.16.178.62:34403)
0806 21:16:17.422367 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:17.422387 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:17.422430 (+ 43us) server_negotiation.cc:407] Connection header received
0806 21:16:17.423155 (+ 725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:17.423178 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:17.423630 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:17.424000 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:17.424459 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:17.425941 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:17.426793 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:17.427512 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:17.427674 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:17.428543 (+ 869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:17.428548 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:17.428563 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:16:17.428982 (+ 419us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:17.429045 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:16:17.429070 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:17.429161 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:17.429312 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 21:16:17.429538 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":130,"threads_started":1}
I20260806 21:16:17.430639 18800 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e07ea765eb7d4e27b48d36029a6efaa2: Neither blocks nor log segments found. Creating new log.
I20260806 21:16:17.431371 18800 log.cc:824] T 00000000000000000000000000000000 P e07ea765eb7d4e27b48d36029a6efaa2: Log is configured to *not* fsync() on all Append() calls
I20260806 21:16:17.433063 17096 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:16:17.433420 18800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e07ea765eb7d4e27b48d36029a6efaa2: No bootstrap required, opened a new log
I20260806 21:16:17.437434 18800 raft_consensus.cc:348] T 00000000000000000000000000000000 P e07ea765eb7d4e27b48d36029a6efaa2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e07ea765eb7d4e27b48d36029a6efaa2" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 34403 } }
I20260806 21:16:17.437718 18800 raft_consensus.cc:374] T 00000000000000000000000000000000 P e07ea765eb7d4e27b48d36029a6efaa2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:16:17.437784 18800 raft_consensus.cc:729] T 00000000000000000000000000000000 P e07ea765eb7d4e27b48d36029a6efaa2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e07ea765eb7d4e27b48d36029a6efaa2, State: Initialized, Role: FOLLOWER
I20260806 21:16:17.438323 18800 consensus_queue.cc:260] T 00000000000000000000000000000000 P e07ea765eb7d4e27b48d36029a6efaa2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e07ea765eb7d4e27b48d36029a6efaa2" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 34403 } }
I20260806 21:16:17.438483 18800 raft_consensus.cc:388] T 00000000000000000000000000000000 P e07ea765eb7d4e27b48d36029a6efaa2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:16:17.438612 18800 raft_consensus.cc:482] T 00000000000000000000000000000000 P e07ea765eb7d4e27b48d36029a6efaa2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:16:17.438750 18800 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e07ea765eb7d4e27b48d36029a6efaa2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:16:17.439054 18803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:17.435056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59913 (local address 127.16.178.62:34403)
0806 21:16:17.435167 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:17.435171 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:17.435355 (+ 184us) server_negotiation.cc:407] Connection header received
0806 21:16:17.435523 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:17.435527 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:17.435573 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:17.435639 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:17.435917 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:17.436502 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:17.437339 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:17.437571 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:17.437662 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:17.438586 (+ 924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:17.438590 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:17.438593 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:16:17.438616 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:17.438650 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:16:17.438666 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:17.438745 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:17.438847 (+ 102us) server_negotiation.cc:299] Negotiation successful
0806 21:16:17.438916 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 21:16:17.439886 18800 raft_consensus.cc:504] T 00000000000000000000000000000000 P e07ea765eb7d4e27b48d36029a6efaa2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e07ea765eb7d4e27b48d36029a6efaa2" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 34403 } }
I20260806 21:16:17.440302 18800 leader_election.cc:302] T 00000000000000000000000000000000 P e07ea765eb7d4e27b48d36029a6efaa2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e07ea765eb7d4e27b48d36029a6efaa2; no voters:
I20260806 21:16:17.440651 18800 leader_election.cc:288] T 00000000000000000000000000000000 P e07ea765eb7d4e27b48d36029a6efaa2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:16:17.440752 18813 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e07ea765eb7d4e27b48d36029a6efaa2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:16:17.440975 18813 raft_consensus.cc:686] T 00000000000000000000000000000000 P e07ea765eb7d4e27b48d36029a6efaa2 [term 1 LEADER]: Becoming Leader. State: Replica: e07ea765eb7d4e27b48d36029a6efaa2, State: Running, Role: LEADER
I20260806 21:16:17.441361 18813 consensus_queue.cc:237] T 00000000000000000000000000000000 P e07ea765eb7d4e27b48d36029a6efaa2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e07ea765eb7d4e27b48d36029a6efaa2" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 34403 } }
I20260806 21:16:17.441946 18800 sys_catalog.cc:564] T 00000000000000000000000000000000 P e07ea765eb7d4e27b48d36029a6efaa2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:16:17.443171 18814 sys_catalog.cc:455] T 00000000000000000000000000000000 P e07ea765eb7d4e27b48d36029a6efaa2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e07ea765eb7d4e27b48d36029a6efaa2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e07ea765eb7d4e27b48d36029a6efaa2" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 34403 } } }
I20260806 21:16:17.443122 18815 sys_catalog.cc:455] T 00000000000000000000000000000000 P e07ea765eb7d4e27b48d36029a6efaa2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e07ea765eb7d4e27b48d36029a6efaa2. Latest consensus state: current_term: 1 leader_uuid: "e07ea765eb7d4e27b48d36029a6efaa2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e07ea765eb7d4e27b48d36029a6efaa2" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 34403 } } }
I20260806 21:16:17.443265 18814 sys_catalog.cc:458] T 00000000000000000000000000000000 P e07ea765eb7d4e27b48d36029a6efaa2 [sys.catalog]: This master's current role is: LEADER
I20260806 21:16:17.443265 18815 sys_catalog.cc:458] T 00000000000000000000000000000000 P e07ea765eb7d4e27b48d36029a6efaa2 [sys.catalog]: This master's current role is: LEADER
I20260806 21:16:17.443754 18825 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:16:17.447216 18825 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:16:17.453966 18825 catalog_manager.cc:1347] Generated new cluster ID: 352284efe7d448b2b489ac1ecb3c883b
I20260806 21:16:17.454052 18825 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:16:17.469173 18825 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:16:17.470165 18825 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:16:17.478434 18825 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e07ea765eb7d4e27b48d36029a6efaa2: Generated new TSK 0
I20260806 21:16:17.479182 18825 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:16:17.486750 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 18688
2026-08-06T21:16:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1570 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050686797304-17096-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:16:17 dist-test-slave-lgr5 krb5kdc[18845](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:16:17 dist-test-slave-lgr5 krb5kdc[18845](info): set up 2 sockets
Aug 06 21:16:17 dist-test-slave-lgr5 krb5kdc[18845](info): commencing operation
krb5kdc: starting...
W20260806 21:16:18.644251 17096 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Aug 06 21:16:18 dist-test-slave-lgr5 krb5kdc[18845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050978, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:16:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:16:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.178.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
I20260806 21:16:18.771369 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:38323
--webserver_interface=127.16.178.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:37695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:38323
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:16:18.889375 18859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:16:18.889714 18859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:16:18.889777 18859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:16:18.893999 18859 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:16:18.894075 18859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:16:18.894104 18859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:16:18.894127 18859 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:16:18.894157 18859 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:16:18.899659 18859 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:37695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:38323
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:38323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.178.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18859
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:16:18.900878 18859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:16:18.901960 18859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:16:18.910552 18874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:16:18.910563 18869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:16:18.910591 18859 server_base.cc:1034] running on GCE node
W20260806 21:16:18.910552 18870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:16:18.911352 18859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:16:18.912547 18859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:16:18.913707 18859 hybrid_clock.cc:648] HybridClock initialized: now 1786050978913695 us; error 28 us; skew 500 ppm
Aug 06 21:16:18 dist-test-slave-lgr5 krb5kdc[18845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050978, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:16:18.916548 18859 init.cc:377] Logged in from keytab as kudu/127.16.178.62@KRBTEST.COM (short username kudu)
I20260806 21:16:18.917490 18859 webserver.cc:460] Webserver started at http://127.16.178.62:38785/ using document root <none> and password file <none>
I20260806 21:16:18.918088 18859 fs_manager.cc:359] Metadata directory not provided
I20260806 21:16:18.918152 18859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:16:18.918341 18859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:16:18.920468 18859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "33320425075f401d83a4a9904a9ccc57"
format_stamp: "Formatted at 2026-08-06 21:16:18 on dist-test-slave-lgr5"
server_key: "1d2b25e32b160962546cd1cc13feb75f"
server_key_iv: "f8975411b1b2cc8777a6854201cf597c"
server_key_version: "encryptionkey@0"
I20260806 21:16:18.921012 18859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "33320425075f401d83a4a9904a9ccc57"
format_stamp: "Formatted at 2026-08-06 21:16:18 on dist-test-slave-lgr5"
server_key: "1d2b25e32b160962546cd1cc13feb75f"
server_key_iv: "f8975411b1b2cc8777a6854201cf597c"
server_key_version: "encryptionkey@0"
I20260806 21:16:18.924784 18859 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 21:16:18.927177 18885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:16:18.928592 18859 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260806 21:16:18.928706 18859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "33320425075f401d83a4a9904a9ccc57"
format_stamp: "Formatted at 2026-08-06 21:16:18 on dist-test-slave-lgr5"
server_key: "1d2b25e32b160962546cd1cc13feb75f"
server_key_iv: "f8975411b1b2cc8777a6854201cf597c"
server_key_version: "encryptionkey@0"
I20260806 21:16:18.928799 18859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:16:18.933451 18859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:16:18.936776 18859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:16:18.936954 18859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:16:18.944136 18859 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.62:38323
I20260806 21:16:18.944166 18965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.62:38323 every 8 connection(s)
I20260806 21:16:18.945163 18859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:16:18.948256 18968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:16:18.948549 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 18859
I20260806 21:16:18.948685 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:16:18.948957 17096 external_mini_cluster.cc:1428] Setting key 37010fc9013c23487e46fbe639d49d75
I20260806 21:16:18.955523 18971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:18.950504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47819 (local address 127.16.178.62:38323)
0806 21:16:18.951058 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:18.951079 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:18.951129 (+ 50us) server_negotiation.cc:407] Connection header received
0806 21:16:18.951990 (+ 861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:18.952021 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:18.952519 (+ 498us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:18.952975 (+ 456us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:18.953856 (+ 881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:18.953863 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:18.953881 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:16:18.953895 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:18.953971 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:16:18.954001 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:18.954161 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:18.954232 (+ 71us) server_negotiation.cc:299] Negotiation successful
0806 21:16:18.954406 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":158,"threads_started":1}
I20260806 21:16:18.957098 18968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33320425075f401d83a4a9904a9ccc57: Bootstrap starting.
I20260806 21:16:18.959482 17096 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:16:18.960251 18968 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33320425075f401d83a4a9904a9ccc57: Neither blocks nor log segments found. Creating new log.
I20260806 21:16:18.961120 18968 log.cc:824] T 00000000000000000000000000000000 P 33320425075f401d83a4a9904a9ccc57: Log is configured to *not* fsync() on all Append() calls
I20260806 21:16:18.963482 18971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:18.961857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44389 (local address 127.16.178.62:38323)
0806 21:16:18.962001 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:18.962006 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:18.962166 (+ 160us) server_negotiation.cc:407] Connection header received
0806 21:16:18.962312 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:18.962314 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:18.962346 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:18.962425 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:18.963117 (+ 692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:18.963120 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:18.963122 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 21:16:18.963133 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:18.963158 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 21:16:18.963168 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:18.963234 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:18.963327 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 21:16:18.963361 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 21:16:18.963837 18968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33320425075f401d83a4a9904a9ccc57: No bootstrap required, opened a new log
I20260806 21:16:18.967533 18968 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33320425075f401d83a4a9904a9ccc57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33320425075f401d83a4a9904a9ccc57" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 38323 } }
I20260806 21:16:18.967792 18968 raft_consensus.cc:374] T 00000000000000000000000000000000 P 33320425075f401d83a4a9904a9ccc57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 21:16:18.967851 18968 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33320425075f401d83a4a9904a9ccc57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33320425075f401d83a4a9904a9ccc57, State: Initialized, Role: FOLLOWER
I20260806 21:16:18.968318 18968 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33320425075f401d83a4a9904a9ccc57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33320425075f401d83a4a9904a9ccc57" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 38323 } }
I20260806 21:16:18.968457 18968 raft_consensus.cc:388] T 00000000000000000000000000000000 P 33320425075f401d83a4a9904a9ccc57 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 21:16:18.968532 18968 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33320425075f401d83a4a9904a9ccc57 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 21:16:18.968634 18968 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33320425075f401d83a4a9904a9ccc57 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:16:18.969615 18968 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33320425075f401d83a4a9904a9ccc57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33320425075f401d83a4a9904a9ccc57" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 38323 } }
I20260806 21:16:18.969965 18968 leader_election.cc:302] T 00000000000000000000000000000000 P 33320425075f401d83a4a9904a9ccc57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 33320425075f401d83a4a9904a9ccc57; no voters:
I20260806 21:16:18.970295 18968 leader_election.cc:288] T 00000000000000000000000000000000 P 33320425075f401d83a4a9904a9ccc57 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 21:16:18.970446 18983 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33320425075f401d83a4a9904a9ccc57 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:16:18.970752 18983 raft_consensus.cc:686] T 00000000000000000000000000000000 P 33320425075f401d83a4a9904a9ccc57 [term 1 LEADER]: Becoming Leader. State: Replica: 33320425075f401d83a4a9904a9ccc57, State: Running, Role: LEADER
I20260806 21:16:18.971065 18983 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33320425075f401d83a4a9904a9ccc57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33320425075f401d83a4a9904a9ccc57" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 38323 } }
I20260806 21:16:18.971606 18968 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33320425075f401d83a4a9904a9ccc57 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:16:18.972826 18984 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33320425075f401d83a4a9904a9ccc57 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "33320425075f401d83a4a9904a9ccc57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33320425075f401d83a4a9904a9ccc57" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 38323 } } }
I20260806 21:16:18.972946 18984 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33320425075f401d83a4a9904a9ccc57 [sys.catalog]: This master's current role is: LEADER
I20260806 21:16:18.973362 18999 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:16:18.973320 18985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33320425075f401d83a4a9904a9ccc57 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33320425075f401d83a4a9904a9ccc57. Latest consensus state: current_term: 1 leader_uuid: "33320425075f401d83a4a9904a9ccc57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33320425075f401d83a4a9904a9ccc57" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 38323 } } }
I20260806 21:16:18.973510 18985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33320425075f401d83a4a9904a9ccc57 [sys.catalog]: This master's current role is: LEADER
I20260806 21:16:18.976354 18999 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:16:18.982661 18999 catalog_manager.cc:1347] Generated new cluster ID: 7c1073d295dc443981da6b64823d67f3
I20260806 21:16:18.982759 18999 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:16:19.004918 18999 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:16:19.005303 18999 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:16:19.024273 18999 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 33320425075f401d83a4a9904a9ccc57: Generated new TSK 0
I20260806 21:16:19.025040 18999 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 21:16:19.033164 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 18859
2026-08-06T21:16:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1540 ms)
[----------] 18 tests from AuthTokenIssuingTest (27967 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:16:19 dist-test-slave-lgr5 krb5kdc[19016](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:16:19 dist-test-slave-lgr5 krb5kdc[19016](info): set up 2 sockets
Aug 06 21:16:19 dist-test-slave-lgr5 krb5kdc[19016](info): commencing operation
krb5kdc: starting...
W20260806 21:16:20.189838 17096 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Aug 06 21:16:20 dist-test-slave-lgr5 krb5kdc[19016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:16:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:16:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.178.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
I20260806 21:16:20.321231 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:42067
--webserver_interface=127.16.178.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:40837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:42067,127.16.178.61:41265,127.16.178.60:45073
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:16:20.436609 19046 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 21:16:20.436982 19046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:16:20.437026 19046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:16:20.437085 19046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:16:20.441087 19046 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:16:20.441151 19046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:16:20.441177 19046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:16:20.441198 19046 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:16:20.441228 19046 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:16:20.446568 19046 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:40837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:42067,127.16.178.61:41265,127.16.178.60:45073
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:42067
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.178.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:16:20.447700 19046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:16:20.448800 19046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:16:20.457825 19060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:16:20.457849 19057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:16:20.457854 19056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:16:20.458273 19046 server_base.cc:1034] running on GCE node
I20260806 21:16:20.458989 19046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:16:20.460316 19046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:16:20.461519 19046 hybrid_clock.cc:648] HybridClock initialized: now 1786050980461504 us; error 31 us; skew 500 ppm
Aug 06 21:16:20 dist-test-slave-lgr5 krb5kdc[19016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050980, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:16:20.468993 19046 init.cc:377] Logged in from keytab as kudu/127.16.178.62@KRBTEST.COM (short username kudu)
I20260806 21:16:20.470006 19046 webserver.cc:460] Webserver started at http://127.16.178.62:44549/ using document root <none> and password file <none>
I20260806 21:16:20.470582 19046 fs_manager.cc:359] Metadata directory not provided
I20260806 21:16:20.470655 19046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:16:20.470875 19046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:16:20.473122 19046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "50d34013489440359e43d0772dde51b6"
format_stamp: "Formatted at 2026-08-06 21:16:20 on dist-test-slave-lgr5"
server_key: "d6c25fe519135badf0fdc09f023b52f4"
server_key_iv: "f6c353a775031130ac656227e6768fbc"
server_key_version: "encryptionkey@0"
I20260806 21:16:20.473687 19046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "50d34013489440359e43d0772dde51b6"
format_stamp: "Formatted at 2026-08-06 21:16:20 on dist-test-slave-lgr5"
server_key: "d6c25fe519135badf0fdc09f023b52f4"
server_key_iv: "f6c353a775031130ac656227e6768fbc"
server_key_version: "encryptionkey@0"
I20260806 21:16:20.477536 19046 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 21:16:20.479949 19072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:16:20.481241 19046 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 21:16:20.481364 19046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "50d34013489440359e43d0772dde51b6"
format_stamp: "Formatted at 2026-08-06 21:16:20 on dist-test-slave-lgr5"
server_key: "d6c25fe519135badf0fdc09f023b52f4"
server_key_iv: "f6c353a775031130ac656227e6768fbc"
server_key_version: "encryptionkey@0"
I20260806 21:16:20.481462 19046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:16:20.499751 19046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:16:20.502899 19046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:16:20.503082 19046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:16:20.509960 19046 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.62:42067
I20260806 21:16:20.510250 19154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.62:42067 every 8 connection(s)
I20260806 21:16:20.510977 19046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:16:20.514014 19155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:16:20.518119 19155 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.178.62" port: 42067 } has no permanent_uuid. Determining permanent_uuid...
I20260806 21:16:20.518783 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 19046
I20260806 21:16:20.518898 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:16:20.519158 17096 external_mini_cluster.cc:1428] Setting key fce875cf33397187dad7eab5281178de
WARNING: no policy specified for kudu/127.16.178.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.61@KRBTEST.COM" created.
Aug 06 21:16:20 dist-test-slave-lgr5 krb5kdc[19016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050980, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:16:20.537046 19158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:20.520595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48676 (local address 127.16.178.62:42067)
0806 21:16:20.521336 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:20.521356 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:20.521513 (+ 157us) server_negotiation.cc:407] Connection header received
0806 21:16:20.522603 (+ 1090us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:20.522610 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:20.522845 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:20.523039 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:20.523853 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:20.525262 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:20.526268 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:20.526865 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:20.527036 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:20.531880 (+ 4844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:20.531886 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:20.531896 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:16:20.531918 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:20.533759 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:20.534246 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:20.534253 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:20.534255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:20.534389 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:20.534729 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:20.534732 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:20.534733 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:20.535296 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:20.535439 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:20.535900 (+ 461us) server_negotiation.cc:299] Negotiation successful
0806 21:16:20.536189 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":102,"server-negotiator.queue_time_us":586,"thread_start_us":169,"threads_started":1}
I20260806 21:16:20.537046 19160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:20.520595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:42067 (local address 127.0.0.1:48676)
0806 21:16:20.521330 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:20.521385 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 21:16:20.522266 (+ 881us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:16:20.523078 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:16:20.523083 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:16:20.523432 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:16:20.523692 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:16:20.523705 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:16:20.525397 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:16:20.525400 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:16:20.526145 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:16:20.526155 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:16:20.527043 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:16:20.527047 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:16:20.527210 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:20.527834 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:16:20.527851 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:16:20.531733 (+ 3882us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:16:20.533879 (+ 2146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:16:20.533888 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:16:20.533896 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:16:20.534124 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:16:20.534511 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:16:20.534513 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:16:20.534515 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:16:20.534634 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:16:20.535454 (+ 820us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:16:20.535477 (+ 23us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:16:20.535633 (+ 156us) client_negotiation.cc:769] Sending connection context
0806 21:16:20.535792 (+ 159us) client_negotiation.cc:240] Negotiation successful
0806 21:16:20.536109 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"mutex_wait_us":46,"spinlock_wait_cycles":69504,"thread_start_us":170,"threads_started":1}
I20260806 21:16:20.539626 19155 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.178.61" port: 41265 } has no permanent_uuid. Determining permanent_uuid...
I20260806 21:16:20.540328 19160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:20.540068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.61:41265 (local address 127.0.0.1:46554)
0806 21:16:20.540202 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:20.540270 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.61:41265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260806 21:16:20.540505 19075 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.178.61:41265: connect: Connection refused (error 111)
I20260806 21:16:20.541286 19160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:20.541096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.61:41265 (local address 127.0.0.1:46556)
0806 21:16:20.541193 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:20.541238 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.61:41265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 21:16:20.541405 19155 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.178.61:41265: Network error: Client connection negotiation failed: client connection to 127.16.178.61:41265: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.61.keytab.
Entry for principal kudu/127.16.178.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.61.keytab.
I20260806 21:16:20.547405 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.61:41265
--webserver_interface=127.16.178.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.61.keytab
--principal=kudu/127.16.178.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:40837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:42067,127.16.178.61:41265,127.16.178.60:45073
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 21:16:20.590647 19155 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.178.61" port: 41265 } attempt: 1
I20260806 21:16:20.591566 19160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:20.591269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.61:41265 (local address 127.0.0.1:46570)
0806 21:16:20.591420 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:20.591483 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.61:41265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260806 21:16:20.592273 19160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:20.592060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.61:41265 (local address 127.0.0.1:46574)
0806 21:16:20.592175 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:20.592218 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.61:41265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 21:16:20.592391 19155 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.178.61:41265: Network error: Client connection negotiation failed: client connection to 127.16.178.61:41265: connect: Connection refused (error 111)
I20260806 21:16:20.660531 19155 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.178.61" port: 41265 } attempt: 2
I20260806 21:16:20.661481 19160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:20.661165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.61:41265 (local address 127.0.0.1:46584)
0806 21:16:20.661331 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:20.661397 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.61:41265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260806 21:16:20.661557 19167 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 21:16:20.661952 19167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:16:20.662009 19167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:16:20.662073 19167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 21:16:20.662302 19160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:20.662062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.61:41265 (local address 127.0.0.1:46590)
0806 21:16:20.662176 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:20.662229 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.61:41265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 21:16:20.662421 19155 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.178.61:41265: Network error: Client connection negotiation failed: client connection to 127.16.178.61:41265: connect: Connection refused (error 111)
W20260806 21:16:20.666466 19167 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:16:20.666543 19167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:16:20.666560 19167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:16:20.666576 19167 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:16:20.666596 19167 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:16:20.672148 19167 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:40837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:42067,127.16.178.61:41265,127.16.178.60:45073
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.61.keytab
--principal=kudu/127.16.178.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.61:41265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.16.178.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:16:20.673241 19167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:16:20.674252 19167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:16:20.683013 19175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:16:20.683141 19173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:16:20.683023 19172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:16:20.683395 19167 server_base.cc:1034] running on GCE node
I20260806 21:16:20.684044 19167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:16:20.685307 19167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:16:20.686504 19167 hybrid_clock.cc:648] HybridClock initialized: now 1786050980686491 us; error 38 us; skew 500 ppm
Aug 06 21:16:20 dist-test-slave-lgr5 krb5kdc[19016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050980, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:16:20.689318 19167 init.cc:377] Logged in from keytab as kudu/127.16.178.61@KRBTEST.COM (short username kudu)
I20260806 21:16:20.690213 19167 webserver.cc:460] Webserver started at http://127.16.178.61:39357/ using document root <none> and password file <none>
I20260806 21:16:20.690760 19167 fs_manager.cc:359] Metadata directory not provided
I20260806 21:16:20.690836 19167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:16:20.691046 19167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:16:20.693137 19167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-1/data/instance:
uuid: "7dd1df313c1843c5aa98a6c0a0304176"
format_stamp: "Formatted at 2026-08-06 21:16:20 on dist-test-slave-lgr5"
server_key: "8d39a5463e399311a6e99844f82808b3"
server_key_iv: "270d73d12e0d724d5a39b37fb5b70f54"
server_key_version: "encryptionkey@0"
I20260806 21:16:20.693703 19167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-1/wal/instance:
uuid: "7dd1df313c1843c5aa98a6c0a0304176"
format_stamp: "Formatted at 2026-08-06 21:16:20 on dist-test-slave-lgr5"
server_key: "8d39a5463e399311a6e99844f82808b3"
server_key_iv: "270d73d12e0d724d5a39b37fb5b70f54"
server_key_version: "encryptionkey@0"
I20260806 21:16:20.697371 19167 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 21:16:20.699710 19187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:16:20.700881 19167 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 21:16:20.701004 19167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-1/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-1/wal
uuid: "7dd1df313c1843c5aa98a6c0a0304176"
format_stamp: "Formatted at 2026-08-06 21:16:20 on dist-test-slave-lgr5"
server_key: "8d39a5463e399311a6e99844f82808b3"
server_key_iv: "270d73d12e0d724d5a39b37fb5b70f54"
server_key_version: "encryptionkey@0"
I20260806 21:16:20.701099 19167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:16:20.726162 19167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:16:20.729100 19167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:16:20.729285 19167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:16:20.736222 19167 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.61:41265
I20260806 21:16:20.736526 19262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.61:41265 every 8 connection(s)
I20260806 21:16:20.737183 19167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-1/data/info.pb
I20260806 21:16:20.740022 19263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:16:20.743927 19263 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.178.62" port: 42067 } has no permanent_uuid. Determining permanent_uuid...
I20260806 21:16:20.744366 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 19167
I20260806 21:16:20.744493 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-1/wal/instance
I20260806 21:16:20.744757 17096 external_mini_cluster.cc:1428] Setting key a7138f6c1413b93b8cc3b26ed2022299
WARNING: no policy specified for kudu/127.16.178.60@KRBTEST.COM; defaulting to no policy
I20260806 21:16:20.753607 19155 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.178.61" port: 41265 } attempt: 3
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.60@KRBTEST.COM" created.
Aug 06 21:16:20 dist-test-slave-lgr5 krb5kdc[19016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050980, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.61@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
Aug 06 21:16:20 dist-test-slave-lgr5 krb5kdc[19016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050980, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for kudu/127.16.178.61@KRBTEST.COM
I20260806 21:16:20.761747 19158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:20.746132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48684 (local address 127.16.178.62:42067)
0806 21:16:20.746274 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:20.746279 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:20.747446 (+ 1167us) server_negotiation.cc:407] Connection header received
0806 21:16:20.748542 (+ 1096us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:20.748546 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:20.748616 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:20.748693 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:20.749995 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:20.750528 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:20.751702 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:20.751953 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:20.752056 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:20.757217 (+ 5161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:20.757222 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:20.757224 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:16:20.757256 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:20.759012 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:20.759699 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:20.759703 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:20.759705 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:20.759794 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:20.760274 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:20.760277 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:20.760279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:20.760526 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:20.760626 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:20.761445 (+ 819us) server_negotiation.cc:299] Negotiation successful
0806 21:16:20.761604 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 21:16:20.763084 19267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:20.746660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:42067 (local address 127.0.0.1:48684)
0806 21:16:20.747241 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:20.747297 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 21:16:20.748310 (+ 1013us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:16:20.748886 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:16:20.748893 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:16:20.749255 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:16:20.749836 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:16:20.749852 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:16:20.750658 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:16:20.750660 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:16:20.751572 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:16:20.751582 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:16:20.752151 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:16:20.752156 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:16:20.752341 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:20.752925 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:16:20.752942 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:16:20.757063 (+ 4121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:16:20.759148 (+ 2085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:16:20.759155 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:16:20.759173 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:16:20.759580 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:16:20.759906 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:16:20.759909 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:16:20.759911 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:16:20.760166 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:16:20.760637 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:16:20.760645 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:16:20.761056 (+ 411us) client_negotiation.cc:769] Sending connection context
0806 21:16:20.761367 (+ 311us) client_negotiation.cc:240] Negotiation successful
0806 21:16:20.761799 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":157,"threads_started":1}
I20260806 21:16:20.765053 19160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:20.754074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.61:41265 (local address 127.0.0.1:46606)
0806 21:16:20.754319 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:20.754332 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:16:20.754403 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:16:20.754991 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:16:20.754994 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:16:20.755188 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:16:20.755384 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:16:20.755391 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:16:20.756532 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:16:20.756536 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:16:20.757098 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:16:20.757109 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:16:20.757910 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:16:20.757914 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:16:20.758038 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:20.758636 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:16:20.758648 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:16:20.760728 (+ 2080us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:16:20.763292 (+ 2564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:16:20.763296 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:16:20.763298 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:16:20.763534 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:16:20.763842 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:16:20.763845 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:16:20.763847 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:16:20.763938 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:16:20.764528 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:16:20.764532 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:16:20.764637 (+ 105us) client_negotiation.cc:769] Sending connection context
0806 21:16:20.764728 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 21:16:20.764886 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 21:16:20.765002 19263 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.178.61" port: 41265 } has no permanent_uuid. Determining permanent_uuid...
I20260806 21:16:20.765188 19268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:20.754223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46606 (local address 127.16.178.61:41265)
0806 21:16:20.754592 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:20.754602 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:20.754622 (+ 20us) server_negotiation.cc:407] Connection header received
0806 21:16:20.754658 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:20.754662 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:20.754837 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:20.754994 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:20.755494 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:20.756344 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:20.757252 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:20.757775 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:20.757891 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:20.760859 (+ 2968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:20.760868 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:20.760882 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:16:20.760903 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:20.763130 (+ 2227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:20.763652 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:20.763659 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:20.763660 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:20.763747 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:20.764062 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:20.764066 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:20.764067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:20.764418 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:20.764546 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:20.764829 (+ 283us) server_negotiation.cc:299] Negotiation successful
0806 21:16:20.764980 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":149,"threads_started":1}
I20260806 21:16:20.767395 19155 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.178.60" port: 45073 } has no permanent_uuid. Determining permanent_uuid...
I20260806 21:16:20.768211 19160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:20.767915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:45073 (local address 127.0.0.1:45912)
0806 21:16:20.768039 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:20.768132 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.60:45073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260806 21:16:20.768967 19160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:20.768673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:45073 (local address 127.0.0.1:45926)
0806 21:16:20.768841 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:20.768895 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.60:45073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260806 21:16:20.769119 19155 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.178.60:45073: Network error: Client connection negotiation failed: client connection to 127.16.178.60:45073: connect: Connection refused (error 111)
Aug 06 21:16:20 dist-test-slave-lgr5 krb5kdc[19016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050980, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.61@KRBTEST.COM for kudu/127.16.178.61@KRBTEST.COM
I20260806 21:16:20.774080 19267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:20.765577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.61:41265 (local address 127.0.0.1:46612)
0806 21:16:20.765831 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:20.765845 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:16:20.765924 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:16:20.766638 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:16:20.766641 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:16:20.766807 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:16:20.766935 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:16:20.766943 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:16:20.767875 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:16:20.767878 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:16:20.768602 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:16:20.768614 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:16:20.769243 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:16:20.769247 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:16:20.769317 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:20.769774 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:16:20.769786 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:16:20.771050 (+ 1264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:16:20.772657 (+ 1607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:16:20.772660 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:16:20.772663 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:16:20.772859 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:16:20.773118 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:16:20.773120 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:16:20.773121 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:16:20.773198 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:16:20.773567 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:16:20.773569 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:16:20.773653 (+ 84us) client_negotiation.cc:769] Sending connection context
0806 21:16:20.773734 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 21:16:20.773884 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176}
I20260806 21:16:20.774111 19268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:20.765813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46612 (local address 127.16.178.61:41265)
0806 21:16:20.765917 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:20.765921 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:20.765933 (+ 12us) server_negotiation.cc:407] Connection header received
0806 21:16:20.766029 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:20.766032 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:20.766076 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:20.766152 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:20.767070 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:20.767753 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:20.768726 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:20.769025 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:20.769126 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:20.771210 (+ 2084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:20.771214 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:20.771216 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:16:20.771241 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:20.772534 (+ 1293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:20.772939 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:20.772942 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:20.772943 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:20.773036 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:20.773266 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:20.773268 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:20.773269 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:20.773479 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:20.773571 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:20.773802 (+ 231us) server_negotiation.cc:299] Negotiation successful
0806 21:16:20.773954 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":42}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.60.keytab.
Entry for principal kudu/127.16.178.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.60.keytab.
I20260806 21:16:20.775040 19263 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.178.60" port: 45073 } has no permanent_uuid. Determining permanent_uuid...
I20260806 21:16:20.775610 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.60:45073
--webserver_interface=127.16.178.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.60.keytab
--principal=kudu/127.16.178.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:40837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:42067,127.16.178.61:41265,127.16.178.60:45073
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 21:16:20.776150 19267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:20.775646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:45073 (local address 127.0.0.1:45928)
0806 21:16:20.775917 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:20.776026 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.60:45073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180}
W20260806 21:16:20.776391 19191 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.178.60:45073: connect: Connection refused (error 111)
I20260806 21:16:20.777426 19267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:20.777183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:45073 (local address 127.0.0.1:45944)
0806 21:16:20.777308 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:20.777356 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.60:45073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 21:16:20.777693 19263 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.178.60:45073: Network error: Client connection negotiation failed: client connection to 127.16.178.60:45073: connect: Connection refused (error 111)
I20260806 21:16:20.800269 19155 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.178.60" port: 45073 } attempt: 1
I20260806 21:16:20.801123 19160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:20.800840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:45073 (local address 127.0.0.1:45960)
0806 21:16:20.800979 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:20.801044 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.60:45073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 21:16:20.801882 19160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:20.801671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:45073 (local address 127.0.0.1:45966)
0806 21:16:20.801766 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:20.801810 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.60:45073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 21:16:20.802011 19155 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.178.60:45073: Network error: Client connection negotiation failed: client connection to 127.16.178.60:45073: connect: Connection refused (error 111)
I20260806 21:16:20.827029 19263 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.178.60" port: 45073 } attempt: 1
I20260806 21:16:20.827940 19267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:20.827641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:45073 (local address 127.0.0.1:45970)
0806 21:16:20.827794 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:20.827857 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.60:45073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260806 21:16:20.829305 19267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:20.829041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:45073 (local address 127.0.0.1:45984)
0806 21:16:20.829178 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:20.829241 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.60:45073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260806 21:16:20.829479 19263 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.178.60:45073: Network error: Client connection negotiation failed: client connection to 127.16.178.60:45073: connect: Connection refused (error 111)
I20260806 21:16:20.877164 19155 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.178.60" port: 45073 } attempt: 2
I20260806 21:16:20.877986 19160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:20.877709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:45073 (local address 127.0.0.1:45986)
0806 21:16:20.877851 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:20.877910 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.60:45073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 21:16:20.878718 19160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:20.878533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:45073 (local address 127.0.0.1:45990)
0806 21:16:20.878622 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:20.878665 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.60:45073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 21:16:20.879379 19155 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.178.60:45073: Network error: Client connection negotiation failed: client connection to 127.16.178.60:45073: connect: Connection refused (error 111)
W20260806 21:16:20.891901 19272 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 21:16:20.892177 19272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:16:20.892218 19272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:16:20.892282 19272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:16:20.896188 19272 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:16:20.896235 19272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:16:20.896262 19272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:16:20.896288 19272 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:16:20.896317 19272 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:16:20.897624 19263 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.178.60" port: 45073 } attempt: 2
I20260806 21:16:20.898296 19267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:20.898051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:45073 (local address 127.0.0.1:45998)
0806 21:16:20.898179 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:20.898229 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.60:45073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 21:16:20.898917 19267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:20.898720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:45073 (local address 127.0.0.1:46010)
0806 21:16:20.898818 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:20.898862 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.60:45073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 21:16:20.899022 19263 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.178.60:45073: Network error: Client connection negotiation failed: client connection to 127.16.178.60:45073: connect: Connection refused (error 111)
I20260806 21:16:20.901655 19272 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:40837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:42067,127.16.178.61:41265,127.16.178.60:45073
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.60.keytab
--principal=kudu/127.16.178.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.60:45073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.16.178.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19272
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:16:20.902631 19272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:16:20.903576 19272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:16:20.911548 19283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:16:20.911558 19288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:16:20.911576 19272 server_base.cc:1034] running on GCE node
W20260806 21:16:20.911669 19284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:16:20.912263 19272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:16:20.913280 19272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:16:20.914441 19272 hybrid_clock.cc:648] HybridClock initialized: now 1786050980914425 us; error 31 us; skew 500 ppm
Aug 06 21:16:20 dist-test-slave-lgr5 krb5kdc[19016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050980, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:16:20.917052 19272 init.cc:377] Logged in from keytab as kudu/127.16.178.60@KRBTEST.COM (short username kudu)
I20260806 21:16:20.917850 19272 webserver.cc:460] Webserver started at http://127.16.178.60:43945/ using document root <none> and password file <none>
I20260806 21:16:20.918326 19272 fs_manager.cc:359] Metadata directory not provided
I20260806 21:16:20.918381 19272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:16:20.918555 19272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:16:20.920581 19272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-2/data/instance:
uuid: "1e2383028b5743898465b277eed3dda9"
format_stamp: "Formatted at 2026-08-06 21:16:20 on dist-test-slave-lgr5"
server_key: "08581a571ec23090fbc747b0db607959"
server_key_iv: "a12e606f65e5612aedb30fb05d516b9a"
server_key_version: "encryptionkey@0"
I20260806 21:16:20.921123 19272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-2/wal/instance:
uuid: "1e2383028b5743898465b277eed3dda9"
format_stamp: "Formatted at 2026-08-06 21:16:20 on dist-test-slave-lgr5"
server_key: "08581a571ec23090fbc747b0db607959"
server_key_iv: "a12e606f65e5612aedb30fb05d516b9a"
server_key_version: "encryptionkey@0"
I20260806 21:16:20.924966 19272 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 21:16:20.927273 19300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:16:20.928531 19272 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 21:16:20.928642 19272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-2/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-2/wal
uuid: "1e2383028b5743898465b277eed3dda9"
format_stamp: "Formatted at 2026-08-06 21:16:20 on dist-test-slave-lgr5"
server_key: "08581a571ec23090fbc747b0db607959"
server_key_iv: "a12e606f65e5612aedb30fb05d516b9a"
server_key_version: "encryptionkey@0"
I20260806 21:16:20.928746 19272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:16:20.953351 19272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:16:20.956607 19272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:16:20.956794 19272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:16:20.964151 19272 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.60:45073
I20260806 21:16:20.964426 19381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.60:45073 every 8 connection(s)
I20260806 21:16:20.965181 19272 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-2/data/info.pb
I20260806 21:16:20.968015 19384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:16:20.971777 19384 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.178.62" port: 42067 } has no permanent_uuid. Determining permanent_uuid...
I20260806 21:16:20.973425 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 19272
I20260806 21:16:20.973544 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786050686797304-17096-0/minicluster-data/master-2/wal/instance
I20260806 21:16:20.973841 17096 external_mini_cluster.cc:1428] Setting key 2272307d34e81abad1ed6d9af14a5373
I20260806 21:16:20.978540 19155 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.178.60" port: 45073 } attempt: 3
Aug 06 21:16:20 dist-test-slave-lgr5 krb5kdc[19016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050980, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.60@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
Aug 06 21:16:20 dist-test-slave-lgr5 krb5kdc[19016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.178.60@KRBTEST.COM
Aug 06 21:16:20 dist-test-slave-lgr5 krb5kdc[19016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050980, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for kudu/127.16.178.60@KRBTEST.COM
I20260806 21:16:20.987323 19158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:20.973700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48686 (local address 127.16.178.62:42067)
0806 21:16:20.973872 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:20.973877 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:20.974798 (+ 921us) server_negotiation.cc:407] Connection header received
0806 21:16:20.975877 (+ 1079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:20.975880 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:20.975930 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:20.976005 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:20.977074 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:20.977861 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:20.979300 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:20.979668 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:20.979755 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:20.983011 (+ 3256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:20.983014 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:20.983016 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:16:20.983044 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:20.984672 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:20.985262 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:20.985266 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:20.985268 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:20.985374 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:20.985797 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:20.985801 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:20.985802 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:20.986070 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:20.986178 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:20.986976 (+ 798us) server_negotiation.cc:299] Negotiation successful
0806 21:16:20.987116 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 21:16:20.988446 19389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:20.974489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46014 (local address 127.16.178.60:45073)
0806 21:16:20.974899 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:20.974910 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:20.974950 (+ 40us) server_negotiation.cc:407] Connection header received
0806 21:16:20.975654 (+ 704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:20.975671 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:20.975883 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:20.976061 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:20.976924 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:20.978114 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:20.979291 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:20.979883 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:20.980053 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:20.983285 (+ 3232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:20.983293 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:20.983310 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:16:20.983336 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:20.985641 (+ 2305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:20.986229 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:20.986236 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:20.986238 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:20.986318 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:20.986616 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:20.986620 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:20.986622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:20.986904 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:20.987120 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:20.987495 (+ 375us) server_negotiation.cc:299] Negotiation successful
0806 21:16:20.987644 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":120,"server-negotiator.queue_time_us":324,"spinlock_wait_cycles":1280,"thread_start_us":146,"threads_started":1}
I20260806 21:16:20.988456 19387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:20.974098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:42067 (local address 127.0.0.1:48686)
0806 21:16:20.974636 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:20.974698 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 21:16:20.975663 (+ 965us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:16:20.976100 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:16:20.976106 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:16:20.976563 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:16:20.976936 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:16:20.976951 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:16:20.977986 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:16:20.977989 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:16:20.979159 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:16:20.979170 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:16:20.979959 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:16:20.979963 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:16:20.980164 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:20.980916 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:16:20.980940 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:16:20.982624 (+ 1684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:16:20.984801 (+ 2177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:16:20.984810 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:16:20.984824 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:16:20.985164 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:16:20.985486 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:16:20.985490 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:16:20.985493 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:16:20.985690 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:16:20.986205 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:16:20.986219 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:16:20.986665 (+ 446us) client_negotiation.cc:769] Sending connection context
0806 21:16:20.986925 (+ 260us) client_negotiation.cc:240] Negotiation successful
0806 21:16:20.987280 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":24,"spinlock_wait_cycles":7424,"thread_start_us":117,"threads_started":1}
I20260806 21:16:20.989976 19384 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.178.61" port: 41265 } has no permanent_uuid. Determining permanent_uuid...
I20260806 21:16:20.990312 19263 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.178.60" port: 45073 } attempt: 3
I20260806 21:16:20.990769 19160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:20.979038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:45073 (local address 127.0.0.1:46028)
0806 21:16:20.979176 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:20.979194 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 21:16:20.979338 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:16:20.979869 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:16:20.979872 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:16:20.980113 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:16:20.980223 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:16:20.980231 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:16:20.981234 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:16:20.981238 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:16:20.981925 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:16:20.981937 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:16:20.982462 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:16:20.982468 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:16:20.982574 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:20.983008 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:16:20.983026 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:16:20.984653 (+ 1627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:16:20.988886 (+ 4233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:16:20.988890 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:16:20.988892 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:16:20.989186 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:16:20.989546 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:16:20.989549 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:16:20.989551 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:16:20.989665 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:16:20.990213 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:16:20.990216 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:16:20.990331 (+ 115us) client_negotiation.cc:769] Sending connection context
0806 21:16:20.990443 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 21:16:20.990598 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"spinlock_wait_cycles":13056}
I20260806 21:16:20.991479 19392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:20.979248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46028 (local address 127.16.178.60:45073)
0806 21:16:20.979635 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:20.979645 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:20.979658 (+ 13us) server_negotiation.cc:407] Connection header received
0806 21:16:20.979690 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:20.979692 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:20.979737 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:20.979832 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:20.980404 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:20.981093 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:20.982042 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:20.982324 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:20.982438 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:20.984871 (+ 2433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:20.984876 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:20.984879 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:16:20.984916 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:20.988723 (+ 3807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:20.989341 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:20.989345 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:20.989347 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:20.989445 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:20.989775 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:20.989779 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:20.989781 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:20.990028 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:20.990145 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:20.991186 (+ 1041us) server_negotiation.cc:299] Negotiation successful
0806 21:16:20.991311 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":131,"threads_started":1}
I20260806 21:16:20.995127 19155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50d34013489440359e43d0772dde51b6: Bootstrap starting.
Aug 06 21:16:20 dist-test-slave-lgr5 krb5kdc[19016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050980, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.61@KRBTEST.COM for kudu/127.16.178.60@KRBTEST.COM
Aug 06 21:16:20 dist-test-slave-lgr5 krb5kdc[19016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:16:20.998358 19155 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50d34013489440359e43d0772dde51b6: Neither blocks nor log segments found. Creating new log.
I20260806 21:16:20.999394 19155 log.cc:824] T 00000000000000000000000000000000 P 50d34013489440359e43d0772dde51b6: Log is configured to *not* fsync() on all Append() calls
Aug 06 21:16:21 dist-test-slave-lgr5 krb5kdc[19016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050980, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.60@KRBTEST.COM for kudu/127.16.178.61@KRBTEST.COM
I20260806 21:16:21.001667 19267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:20.990733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:45073 (local address 127.0.0.1:46038)
0806 21:16:20.990909 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:20.990925 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:16:20.991014 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:16:20.991335 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:16:20.991339 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:16:20.991539 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:16:20.991640 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:16:20.991649 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:16:20.993330 (+ 1681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:16:20.993333 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:16:20.994042 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:16:20.994052 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:16:20.994471 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:16:20.994475 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:16:20.994552 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:20.995338 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:16:20.995349 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:16:20.996787 (+ 1438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:16:20.999473 (+ 2686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:16:20.999477 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:16:20.999479 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:16:20.999727 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:16:21.000059 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:16:21.000062 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:16:21.000064 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:16:21.000197 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:16:21.000624 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:16:21.000627 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:16:21.000785 (+ 158us) client_negotiation.cc:769] Sending connection context
0806 21:16:21.001123 (+ 338us) client_negotiation.cc:240] Negotiation successful
0806 21:16:21.001281 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107}
I20260806 21:16:21.001946 19158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:20.992123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48690 (local address 127.16.178.62:42067)
0806 21:16:20.992234 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:20.992237 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:20.992247 (+ 10us) server_negotiation.cc:407] Connection header received
0806 21:16:20.992280 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:20.992282 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:20.992323 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:20.992403 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:20.993095 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:20.993749 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:20.994826 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:20.995121 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:20.995200 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:20.998115 (+ 2915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:20.998119 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:20.998122 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:16:20.998149 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:20.999854 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:21.000424 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:21.000428 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:21.000430 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:21.000524 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:21.000874 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:21.000878 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:21.000879 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:21.001100 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:21.001284 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:21.001664 (+ 380us) server_negotiation.cc:299] Negotiation successful
0806 21:16:21.001785 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 21:16:21.002012 19155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50d34013489440359e43d0772dde51b6: No bootstrap required, opened a new log
I20260806 21:16:21.002709 19393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:20.990760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46038 (local address 127.16.178.60:45073)
0806 21:16:20.991079 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:20.991090 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:20.991104 (+ 14us) server_negotiation.cc:407] Connection header received
0806 21:16:20.991142 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:20.991145 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:20.991187 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:20.991758 (+ 571us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:20.992477 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:20.993204 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:20.994146 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:20.994369 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:20.994457 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:20.996943 (+ 2486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:20.996947 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:20.996950 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:16:20.996975 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:20.999352 (+ 2377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:20.999849 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:20.999853 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:20.999855 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:20.999952 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:21.000314 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:21.000317 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:21.000319 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:21.000533 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:21.000729 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:21.002403 (+ 1674us) server_negotiation.cc:299] Negotiation successful
0806 21:16:21.002520 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":16,"server-negotiator.queue_time_us":263,"spinlock_wait_cycles":1280896,"thread_start_us":133,"threads_started":1}
I20260806 21:16:21.004650 19268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:20.990638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46626 (local address 127.16.178.61:41265)
0806 21:16:20.990765 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:20.990780 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:20.991094 (+ 314us) server_negotiation.cc:407] Connection header received
0806 21:16:20.991269 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:20.991272 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:20.991320 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:20.991373 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:20.996271 (+ 4898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:20.996946 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:20.997989 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:20.998331 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:20.998423 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:21.000609 (+ 2186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:21.000613 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:21.000616 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:16:21.000638 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:21.002708 (+ 2070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:21.003152 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:21.003155 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:21.003157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:21.003238 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:21.003535 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:21.003538 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:21.003540 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:21.003826 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:21.003920 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:21.004356 (+ 436us) server_negotiation.cc:299] Negotiation successful
0806 21:16:21.004498 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 21:16:21.005151 19387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:20.990532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.61:41265 (local address 127.0.0.1:46626)
0806 21:16:20.991023 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:20.991039 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:16:20.991156 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:16:20.991536 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:16:20.991539 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:16:20.991698 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:16:20.992446 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:16:20.992455 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:16:20.997064 (+ 4609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:16:20.997067 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:16:20.997851 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:16:20.997863 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:16:20.998578 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:16:20.998581 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:16:20.998694 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:20.999074 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:16:20.999094 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:16:21.000461 (+ 1367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:16:21.002832 (+ 2371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:16:21.002835 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:16:21.002837 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:16:21.003053 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:16:21.003320 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:16:21.003323 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:16:21.003325 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:16:21.003424 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:16:21.004003 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:16:21.004006 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:16:21.004134 (+ 128us) client_negotiation.cc:769] Sending connection context
0806 21:16:21.004785 (+ 651us) client_negotiation.cc:240] Negotiation successful
0806 21:16:21.004929 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"mutex_wait_us":18}
I20260806 21:16:21.006659 19263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7dd1df313c1843c5aa98a6c0a0304176: Bootstrap starting.
I20260806 21:16:21.006523 19155 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50d34013489440359e43d0772dde51b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d34013489440359e43d0772dde51b6" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 42067 } } peers { permanent_uuid: "7dd1df313c1843c5aa98a6c0a0304176" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 41265 } } peers { permanent_uuid: "1e2383028b5743898465b277eed3dda9" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 45073 } }
I20260806 21:16:21.006805 19155 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50d34013489440359e43d0772dde51b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50d34013489440359e43d0772dde51b6, State: Initialized, Role: FOLLOWER
I20260806 21:16:21.007157 19155 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50d34013489440359e43d0772dde51b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d34013489440359e43d0772dde51b6" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 42067 } } peers { permanent_uuid: "7dd1df313c1843c5aa98a6c0a0304176" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 41265 } } peers { permanent_uuid: "1e2383028b5743898465b277eed3dda9" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 45073 } }
I20260806 21:16:21.007750 19395 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50d34013489440359e43d0772dde51b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d34013489440359e43d0772dde51b6" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 42067 } } peers { permanent_uuid: "7dd1df313c1843c5aa98a6c0a0304176" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 41265 } } peers { permanent_uuid: "1e2383028b5743898465b277eed3dda9" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 45073 } } }
I20260806 21:16:21.007892 19395 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50d34013489440359e43d0772dde51b6 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:16:21.008786 19155 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50d34013489440359e43d0772dde51b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:16:21.008894 19384 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.178.60" port: 45073 } has no permanent_uuid. Determining permanent_uuid...
I20260806 21:16:21.009945 19263 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7dd1df313c1843c5aa98a6c0a0304176: Neither blocks nor log segments found. Creating new log.
I20260806 21:16:21.010977 19263 log.cc:824] T 00000000000000000000000000000000 P 7dd1df313c1843c5aa98a6c0a0304176: Log is configured to *not* fsync() on all Append() calls
Aug 06 21:16:21 dist-test-slave-lgr5 krb5kdc[19016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050980, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.60@KRBTEST.COM for kudu/127.16.178.60@KRBTEST.COM
Aug 06 21:16:21 dist-test-slave-lgr5 krb5kdc[19016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.178.61@KRBTEST.COM
W20260806 21:16:21.015494 19416 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 50d34013489440359e43d0772dde51b6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 21:16:21.015573 19416 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 21:16:21.015650 19263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7dd1df313c1843c5aa98a6c0a0304176: No bootstrap required, opened a new log
I20260806 21:16:21.014153 19046 master_runner.cc:186] Error getting master registration for 127.16.178.61:41265: OK, instance_id { permanent_uuid: "7dd1df313c1843c5aa98a6c0a0304176" instance_seqno: 1786050980734292 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 21:16:21.017624 19046 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 21:16:21.019418 19393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:21.009567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46050 (local address 127.16.178.60:45073)
0806 21:16:21.009725 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:21.009729 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:21.009831 (+ 102us) server_negotiation.cc:407] Connection header received
0806 21:16:21.009974 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:21.009977 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:21.010012 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:21.010051 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:21.010489 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:21.011344 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:21.012341 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:21.012698 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:21.012787 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:21.015740 (+ 2953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:21.015744 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:21.015747 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:16:21.015770 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:21.017350 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:21.017776 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:21.017780 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:21.017782 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:21.017849 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:21.018414 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:21.018419 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:21.018421 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:21.018680 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:21.018801 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:21.019098 (+ 297us) server_negotiation.cc:299] Negotiation successful
0806 21:16:21.019223 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260806 21:16:21.019903 19387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:21.009371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:45073 (local address 127.0.0.1:46050)
0806 21:16:21.009786 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:21.009798 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 21:16:21.009894 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:16:21.010153 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:16:21.010156 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:16:21.010316 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:16:21.010393 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:16:21.010398 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:16:21.011461 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:16:21.011464 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:16:21.012230 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:16:21.012240 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:16:21.012933 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:16:21.012936 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:16:21.013051 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:21.013491 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:16:21.013506 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:16:21.015593 (+ 2087us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:16:21.017458 (+ 1865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:16:21.017461 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:16:21.017463 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:16:21.017688 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:16:21.018201 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:16:21.018204 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:16:21.018206 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:16:21.018290 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:16:21.018808 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:16:21.018812 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:16:21.018928 (+ 116us) client_negotiation.cc:769] Sending connection context
0806 21:16:21.019624 (+ 696us) client_negotiation.cc:240] Negotiation successful
0806 21:16:21.019751 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371}
I20260806 21:16:21.021874 19263 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7dd1df313c1843c5aa98a6c0a0304176 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d34013489440359e43d0772dde51b6" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 42067 } } peers { permanent_uuid: "7dd1df313c1843c5aa98a6c0a0304176" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 41265 } } peers { permanent_uuid: "1e2383028b5743898465b277eed3dda9" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 45073 } }
I20260806 21:16:21.022163 19263 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7dd1df313c1843c5aa98a6c0a0304176 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd1df313c1843c5aa98a6c0a0304176, State: Initialized, Role: FOLLOWER
I20260806 21:16:21.022552 19263 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7dd1df313c1843c5aa98a6c0a0304176 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d34013489440359e43d0772dde51b6" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 42067 } } peers { permanent_uuid: "7dd1df313c1843c5aa98a6c0a0304176" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 41265 } } peers { permanent_uuid: "1e2383028b5743898465b277eed3dda9" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 45073 } }
I20260806 21:16:21.023147 19422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7dd1df313c1843c5aa98a6c0a0304176 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d34013489440359e43d0772dde51b6" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 42067 } } peers { permanent_uuid: "7dd1df313c1843c5aa98a6c0a0304176" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 41265 } } peers { permanent_uuid: "1e2383028b5743898465b277eed3dda9" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 45073 } } }
I20260806 21:16:21.023357 19384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e2383028b5743898465b277eed3dda9: Bootstrap starting.
I20260806 21:16:21.023382 19422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7dd1df313c1843c5aa98a6c0a0304176 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:16:21.024075 19263 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7dd1df313c1843c5aa98a6c0a0304176 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:16:21.026443 19384 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e2383028b5743898465b277eed3dda9: Neither blocks nor log segments found. Creating new log.
I20260806 21:16:21.027489 19384 log.cc:824] T 00000000000000000000000000000000 P 1e2383028b5743898465b277eed3dda9: Log is configured to *not* fsync() on all Append() calls
I20260806 21:16:21.028532 19268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:21.004834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46640 (local address 127.16.178.61:41265)
0806 21:16:21.005255 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:21.005260 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:21.008548 (+ 3288us) server_negotiation.cc:407] Connection header received
0806 21:16:21.008740 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:21.008743 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:21.008791 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:21.008853 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:21.009761 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:21.010479 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:21.011551 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:21.011873 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:21.011945 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:21.018171 (+ 6226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:21.018175 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:21.018178 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:16:21.018205 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:21.019741 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:21.026887 (+ 7146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:21.026892 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:21.026894 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:21.027001 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:21.027379 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:21.027383 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:21.027385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:21.027621 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:21.027790 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:21.028232 (+ 442us) server_negotiation.cc:299] Negotiation successful
0806 21:16:21.028360 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335}
I20260806 21:16:21.029819 19384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e2383028b5743898465b277eed3dda9: No bootstrap required, opened a new log
I20260806 21:16:21.033666 17096 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:16:21.034456 19384 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e2383028b5743898465b277eed3dda9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d34013489440359e43d0772dde51b6" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 42067 } } peers { permanent_uuid: "7dd1df313c1843c5aa98a6c0a0304176" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 41265 } } peers { permanent_uuid: "1e2383028b5743898465b277eed3dda9" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 45073 } }
I20260806 21:16:21.034657 19384 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e2383028b5743898465b277eed3dda9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e2383028b5743898465b277eed3dda9, State: Initialized, Role: FOLLOWER
I20260806 21:16:21.034967 19384 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e2383028b5743898465b277eed3dda9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d34013489440359e43d0772dde51b6" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 42067 } } peers { permanent_uuid: "7dd1df313c1843c5aa98a6c0a0304176" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 41265 } } peers { permanent_uuid: "1e2383028b5743898465b277eed3dda9" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 45073 } }
I20260806 21:16:21.035166 19113 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "50d34013489440359e43d0772dde51b6"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48690
I20260806 21:16:21.035512 19113 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50d34013489440359e43d0772dde51b6 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 21:16:21.035642 19113 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50d34013489440359e43d0772dde51b6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:16:21.036603 19384 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e2383028b5743898465b277eed3dda9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:16:21.037108 19113 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50d34013489440359e43d0772dde51b6 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d34013489440359e43d0772dde51b6" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 42067 } } peers { permanent_uuid: "7dd1df313c1843c5aa98a6c0a0304176" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 41265 } } peers { permanent_uuid: "1e2383028b5743898465b277eed3dda9" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 45073 } }
I20260806 21:16:21.038609 19225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "50d34013489440359e43d0772dde51b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7dd1df313c1843c5aa98a6c0a0304176"
I20260806 21:16:21.038925 19225 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7dd1df313c1843c5aa98a6c0a0304176 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:16:21.038734 19113 leader_election.cc:288] T 00000000000000000000000000000000 P 50d34013489440359e43d0772dde51b6 [CANDIDATE]: Term 1 election: Requested vote from peers 7dd1df313c1843c5aa98a6c0a0304176 (127.16.178.61:41265), 1e2383028b5743898465b277eed3dda9 (127.16.178.60:45073)
I20260806 21:16:21.038980 19345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "50d34013489440359e43d0772dde51b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1e2383028b5743898465b277eed3dda9"
I20260806 21:16:21.040323 19345 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e2383028b5743898465b277eed3dda9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:16:21.036180 19167 master_runner.cc:186] Error getting master registration for 127.16.178.60:45073: OK, instance_id { permanent_uuid: "1e2383028b5743898465b277eed3dda9" instance_seqno: 1786050980962065 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 21:16:21.040854 19167 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 21:16:21.040926 19225 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7dd1df313c1843c5aa98a6c0a0304176 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50d34013489440359e43d0772dde51b6 in term 1.
I20260806 21:16:21.041425 19075 leader_election.cc:302] T 00000000000000000000000000000000 P 50d34013489440359e43d0772dde51b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50d34013489440359e43d0772dde51b6, 7dd1df313c1843c5aa98a6c0a0304176; no voters:
W20260806 21:16:21.041571 19436 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7dd1df313c1843c5aa98a6c0a0304176: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 21:16:21.041640 19436 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 21:16:21.041652 19345 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1e2383028b5743898465b277eed3dda9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50d34013489440359e43d0772dde51b6 in term 1.
I20260806 21:16:21.041771 19395 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50d34013489440359e43d0772dde51b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:16:21.041947 19442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e2383028b5743898465b277eed3dda9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d34013489440359e43d0772dde51b6" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 42067 } } peers { permanent_uuid: "7dd1df313c1843c5aa98a6c0a0304176" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 41265 } } peers { permanent_uuid: "1e2383028b5743898465b277eed3dda9" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 45073 } } }
I20260806 21:16:21.042030 19395 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50d34013489440359e43d0772dde51b6 [term 1 LEADER]: Becoming Leader. State: Replica: 50d34013489440359e43d0772dde51b6, State: Running, Role: LEADER
I20260806 21:16:21.042090 19442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e2383028b5743898465b277eed3dda9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:16:21.042416 19395 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50d34013489440359e43d0772dde51b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d34013489440359e43d0772dde51b6" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 42067 } } peers { permanent_uuid: "7dd1df313c1843c5aa98a6c0a0304176" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 41265 } } peers { permanent_uuid: "1e2383028b5743898465b277eed3dda9" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 45073 } }
I20260806 21:16:21.044440 19395 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50d34013489440359e43d0772dde51b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50d34013489440359e43d0772dde51b6. Latest consensus state: current_term: 1 leader_uuid: "50d34013489440359e43d0772dde51b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d34013489440359e43d0772dde51b6" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 42067 } } peers { permanent_uuid: "7dd1df313c1843c5aa98a6c0a0304176" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 41265 } } peers { permanent_uuid: "1e2383028b5743898465b277eed3dda9" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 45073 } } }
I20260806 21:16:21.044565 19395 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50d34013489440359e43d0772dde51b6 [sys.catalog]: This master's current role is: LEADER
I20260806 21:16:21.047173 19473 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:16:21.048413 19473 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260806 21:16:21.051666 19460 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1e2383028b5743898465b277eed3dda9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260806 21:16:21.056195 19393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:21.043261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46062 (local address 127.16.178.60:45073)
0806 21:16:21.043391 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:21.043395 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:21.044245 (+ 850us) server_negotiation.cc:407] Connection header received
0806 21:16:21.044536 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:21.044540 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:21.044595 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:21.044649 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:21.045732 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:21.046453 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:21.049266 (+ 2813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:21.049593 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:21.049680 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:21.051664 (+ 1984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:21.051669 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:21.051672 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:16:21.051707 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:21.053838 (+ 2131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:21.054634 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:21.054638 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:21.054640 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:21.054744 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:21.055065 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:21.055070 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:21.055071 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:21.055324 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:21.055451 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:21.055831 (+ 380us) server_negotiation.cc:299] Negotiation successful
0806 21:16:21.055982 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260806 21:16:21.058533 19460 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 21:16:21.061380 19158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:21.044816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48698 (local address 127.16.178.62:42067)
0806 21:16:21.044939 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:21.044943 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:21.051840 (+ 6897us) server_negotiation.cc:407] Connection header received
0806 21:16:21.052216 (+ 376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:21.052220 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:21.052275 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:21.052382 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:21.053314 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:21.054090 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:21.055213 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:21.055537 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:21.055613 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:21.057391 (+ 1778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:21.057395 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:21.057397 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:16:21.057428 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:21.059312 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:21.059827 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:21.059830 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:21.059832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:21.059925 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:21.060266 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:21.060270 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:21.060272 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:21.060520 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:21.060632 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:21.061034 (+ 402us) server_negotiation.cc:299] Negotiation successful
0806 21:16:21.061201 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 21:16:21.065696 19268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:21.056564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46652 (local address 127.16.178.61:41265)
0806 21:16:21.056709 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:21.056714 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:21.057051 (+ 337us) server_negotiation.cc:407] Connection header received
0806 21:16:21.057249 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:21.057252 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:21.057315 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:21.057367 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:21.058150 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:21.058934 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:21.060330 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:21.060673 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:21.060751 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:21.062537 (+ 1786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:21.062542 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:21.062545 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:16:21.062579 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:21.064039 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:21.064501 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:21.064504 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:21.064507 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:21.064592 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:21.064847 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:21.064851 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:21.064852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:21.065055 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:21.065195 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:21.065420 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 21:16:21.065533 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 21:16:21.072190 19345 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1e2383028b5743898465b277eed3dda9 [term 1 FOLLOWER]: Refusing update from remote peer 50d34013489440359e43d0772dde51b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:16:21.072615 19225 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7dd1df313c1843c5aa98a6c0a0304176 [term 1 FOLLOWER]: Refusing update from remote peer 50d34013489440359e43d0772dde51b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:16:21.073119 19395 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 50d34013489440359e43d0772dde51b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e2383028b5743898465b277eed3dda9" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 45073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:16:21.073755 19467 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 50d34013489440359e43d0772dde51b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dd1df313c1843c5aa98a6c0a0304176" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 41265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:16:21.078078 19422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7dd1df313c1843c5aa98a6c0a0304176 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50d34013489440359e43d0772dde51b6. Latest consensus state: current_term: 1 leader_uuid: "50d34013489440359e43d0772dde51b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d34013489440359e43d0772dde51b6" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 42067 } } peers { permanent_uuid: "7dd1df313c1843c5aa98a6c0a0304176" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 41265 } } peers { permanent_uuid: "1e2383028b5743898465b277eed3dda9" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 45073 } } }
I20260806 21:16:21.078218 19422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7dd1df313c1843c5aa98a6c0a0304176 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:16:21.083559 19442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e2383028b5743898465b277eed3dda9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50d34013489440359e43d0772dde51b6. Latest consensus state: current_term: 1 leader_uuid: "50d34013489440359e43d0772dde51b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d34013489440359e43d0772dde51b6" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 42067 } } peers { permanent_uuid: "7dd1df313c1843c5aa98a6c0a0304176" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 41265 } } peers { permanent_uuid: "1e2383028b5743898465b277eed3dda9" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 45073 } } }
I20260806 21:16:21.083696 19442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e2383028b5743898465b277eed3dda9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:16:21.094555 19395 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50d34013489440359e43d0772dde51b6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "50d34013489440359e43d0772dde51b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d34013489440359e43d0772dde51b6" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 42067 } } peers { permanent_uuid: "7dd1df313c1843c5aa98a6c0a0304176" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 41265 } } peers { permanent_uuid: "1e2383028b5743898465b277eed3dda9" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 45073 } } }
I20260806 21:16:21.094717 19395 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50d34013489440359e43d0772dde51b6 [sys.catalog]: This master's current role is: LEADER
I20260806 21:16:21.096060 19442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e2383028b5743898465b277eed3dda9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "50d34013489440359e43d0772dde51b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d34013489440359e43d0772dde51b6" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 42067 } } peers { permanent_uuid: "7dd1df313c1843c5aa98a6c0a0304176" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 41265 } } peers { permanent_uuid: "1e2383028b5743898465b277eed3dda9" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 45073 } } }
I20260806 21:16:21.096217 19442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e2383028b5743898465b277eed3dda9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:16:21.099334 19467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50d34013489440359e43d0772dde51b6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "50d34013489440359e43d0772dde51b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d34013489440359e43d0772dde51b6" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 42067 } } peers { permanent_uuid: "7dd1df313c1843c5aa98a6c0a0304176" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 41265 } } peers { permanent_uuid: "1e2383028b5743898465b277eed3dda9" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 45073 } } }
I20260806 21:16:21.099462 19467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50d34013489440359e43d0772dde51b6 [sys.catalog]: This master's current role is: LEADER
I20260806 21:16:21.100765 19422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7dd1df313c1843c5aa98a6c0a0304176 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "50d34013489440359e43d0772dde51b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d34013489440359e43d0772dde51b6" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 42067 } } peers { permanent_uuid: "7dd1df313c1843c5aa98a6c0a0304176" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 41265 } } peers { permanent_uuid: "1e2383028b5743898465b277eed3dda9" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 45073 } } }
I20260806 21:16:21.100888 19422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7dd1df313c1843c5aa98a6c0a0304176 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:16:21.104338 19481 mvcc.cc:205] Tried to move back new op lower bound from 7315664818461859840 to 7315664818353537024. Current Snapshot: MvccSnapshot[applied={T|T < 7315664818461859840}]
I20260806 21:16:21.104622 19479 mvcc.cc:205] Tried to move back new op lower bound from 7315664818461859840 to 7315664818353537024. Current Snapshot: MvccSnapshot[applied={T|T < 7315664818461859840}]
I20260806 21:16:21.110054 19473 catalog_manager.cc:1347] Generated new cluster ID: 7f5458e459004b19a00986ef75a6dd06
I20260806 21:16:21.110131 19473 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:16:21.132572 19473 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:16:21.134007 19473 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:16:21.163445 19473 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 50d34013489440359e43d0772dde51b6: Generated new TSK 0
I20260806 21:16:21.164508 19473 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 21:16:21.204528 19268 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:16:21.201104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46666 (local address 127.16.178.61:41265)
0806 21:16:21.201246 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:21.201251 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:21.203175 (+ 1924us) server_negotiation.cc:407] Connection header received
0806 21:16:21.203407 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:21.203412 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:21.203472 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:21.203536 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:21.204308 (+ 772us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46666: BlockingRecv error: recv got EOF from 127.0.0.1:46666 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260806 21:16:21.206281 19393 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:16:21.201652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46078 (local address 127.16.178.60:45073)
0806 21:16:21.201788 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:21.201793 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:21.205245 (+ 3452us) server_negotiation.cc:407] Connection header received
0806 21:16:21.205453 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:21.205456 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:21.205510 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:21.205562 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:21.206161 (+ 599us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46078: BlockingRecv error: recv got EOF from 127.0.0.1:46078 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260806 21:16:21.207784 19158 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:16:21.201875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48702 (local address 127.16.178.62:42067)
0806 21:16:21.201998 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:21.202003 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:21.206873 (+ 4870us) server_negotiation.cc:407] Connection header received
0806 21:16:21.207046 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:21.207049 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:21.207105 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:21.207153 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:21.207658 (+ 505us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48702: BlockingRecv error: recv got EOF from 127.0.0.1:48702 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260806 21:16:21.212849 19268 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:16:21.211144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46682 (local address 127.16.178.61:41265)
0806 21:16:21.211252 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:21.211257 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:21.211781 (+ 524us) server_negotiation.cc:407] Connection header received
0806 21:16:21.212003 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:21.212006 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:21.212150 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:21.212205 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:21.212743 (+ 538us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46682: BlockingRecv error: recv got EOF from 127.0.0.1:46682 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260806 21:16:21.218247 19268 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:16:21.216628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46684 (local address 127.16.178.61:41265)
0806 21:16:21.216735 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:21.216739 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:21.217300 (+ 561us) server_negotiation.cc:407] Connection header received
0806 21:16:21.217513 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:21.217517 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:21.217569 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:21.217629 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:21.218138 (+ 509us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46684: BlockingRecv error: recv got EOF from 127.0.0.1:46684 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260806 21:16:21.224457 19268 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:16:21.222685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46688 (local address 127.16.178.61:41265)
0806 21:16:21.222807 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:21.222812 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:21.223430 (+ 618us) server_negotiation.cc:407] Connection header received
0806 21:16:21.223642 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:21.223646 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:21.223700 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:21.223754 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:21.224340 (+ 586us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46688: BlockingRecv error: recv got EOF from 127.0.0.1:46688 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260806 21:16:21.233484 19268 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:16:21.231310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46692 (local address 127.16.178.61:41265)
0806 21:16:21.231502 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:21.231507 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:21.232305 (+ 798us) server_negotiation.cc:407] Connection header received
0806 21:16:21.232575 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:21.232579 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:21.232639 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:21.232697 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:21.233352 (+ 655us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46692: BlockingRecv error: recv got EOF from 127.0.0.1:46692 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
W20260806 21:16:21.248262 19268 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:16:21.246283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46696 (local address 127.16.178.61:41265)
0806 21:16:21.246431 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:21.246437 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:21.247092 (+ 655us) server_negotiation.cc:407] Connection header received
0806 21:16:21.247361 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:21.247365 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:21.247433 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:21.247487 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:21.248133 (+ 646us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46696: BlockingRecv error: recv got EOF from 127.0.0.1:46696 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260806 21:16:21.268133 19268 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:16:21.266934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46702 (local address 127.16.178.61:41265)
0806 21:16:21.267085 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:21.267089 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:21.267201 (+ 112us) server_negotiation.cc:407] Connection header received
0806 21:16:21.267335 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:21.267337 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:21.267424 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:21.267524 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:21.267971 (+ 447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46702: BlockingRecv error: recv got EOF from 127.0.0.1:46702 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260806 21:16:21.303369 19268 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:16:21.302197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46712 (local address 127.16.178.61:41265)
0806 21:16:21.302345 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:21.302349 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:21.302461 (+ 112us) server_negotiation.cc:407] Connection header received
0806 21:16:21.302608 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:21.302610 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:21.302664 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:21.302749 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:21.303258 (+ 509us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46712: BlockingRecv error: recv got EOF from 127.0.0.1:46712 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260806 21:16:21.370718 19268 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:16:21.369574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46716 (local address 127.16.178.61:41265)
0806 21:16:21.369708 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:21.369711 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:21.369816 (+ 105us) server_negotiation.cc:407] Connection header received
0806 21:16:21.369944 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:21.369947 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:21.370010 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:21.370093 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:21.370586 (+ 493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46716: BlockingRecv error: recv got EOF from 127.0.0.1:46716 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260806 21:16:21.501933 19268 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:16:21.500807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46730 (local address 127.16.178.61:41265)
0806 21:16:21.500964 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:21.500969 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:21.501090 (+ 121us) server_negotiation.cc:407] Connection header received
0806 21:16:21.501226 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:21.501228 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:21.501275 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:21.501348 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:21.501788 (+ 440us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46730: BlockingRecv error: recv got EOF from 127.0.0.1:46730 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260806 21:16:21.761484 19268 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:16:21.760369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46732 (local address 127.16.178.61:41265)
0806 21:16:21.760524 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:21.760530 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:21.760601 (+ 71us) server_negotiation.cc:407] Connection header received
0806 21:16:21.760756 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:21.760760 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:21.760809 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:21.760878 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:21.761353 (+ 475us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46732: BlockingRecv error: recv got EOF from 127.0.0.1:46732 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 21:16:22.043064 19436 catalog_manager.cc:1259] Loaded cluster ID: 7f5458e459004b19a00986ef75a6dd06
I20260806 21:16:22.043157 19436 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7dd1df313c1843c5aa98a6c0a0304176: loading cluster ID for follower catalog manager: success
I20260806 21:16:22.045006 19436 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7dd1df313c1843c5aa98a6c0a0304176: acquiring CA information for follower catalog manager: success
I20260806 21:16:22.046218 19436 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7dd1df313c1843c5aa98a6c0a0304176: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 21:16:22.059943 19460 catalog_manager.cc:1259] Loaded cluster ID: 7f5458e459004b19a00986ef75a6dd06
I20260806 21:16:22.060030 19460 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1e2383028b5743898465b277eed3dda9: loading cluster ID for follower catalog manager: success
I20260806 21:16:22.061714 19460 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1e2383028b5743898465b277eed3dda9: acquiring CA information for follower catalog manager: success
I20260806 21:16:22.062837 19460 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1e2383028b5743898465b277eed3dda9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 21:16:22.280381 19577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:22.275821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46740 (local address 127.16.178.61:41265)
0806 21:16:22.276150 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:22.276155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:22.276180 (+ 25us) server_negotiation.cc:407] Connection header received
0806 21:16:22.276314 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:22.276318 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:22.276556 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:22.276741 (+ 185us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:16:22.277201 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:22.277795 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:22.278676 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:22.278899 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:22.278972 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:22.279402 (+ 430us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:16:22.279901 (+ 499us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:16:22.280013 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:22.280170 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 21:16:22.280238 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":82,"threads_started":1}
I20260806 21:16:22.287715 19588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:22.283766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46092 (local address 127.16.178.60:45073)
0806 21:16:22.284020 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:22.284024 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:22.284040 (+ 16us) server_negotiation.cc:407] Connection header received
0806 21:16:22.284159 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:22.284162 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:22.284283 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:22.284393 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:16:22.284746 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:22.285313 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:22.286225 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:22.286485 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:22.286577 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:22.286834 (+ 257us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:16:22.287289 (+ 455us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:16:22.287383 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:22.287500 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 21:16:22.287552 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":79,"threads_started":1}
I20260806 21:16:22.294745 19577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:22.290978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46750 (local address 127.16.178.61:41265)
0806 21:16:22.291098 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:22.291103 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:22.291157 (+ 54us) server_negotiation.cc:407] Connection header received
0806 21:16:22.291280 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:22.291283 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:22.291376 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:22.291453 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:16:22.291734 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:22.292403 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:22.293574 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:22.293822 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:22.293912 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:22.294206 (+ 294us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:16:22.294290 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:16:22.294378 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:22.294537 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 21:16:22.294595 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 21:16:22.294773 19588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:22.290748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46102 (local address 127.16.178.60:45073)
0806 21:16:22.290850 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:22.290854 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:22.291460 (+ 606us) server_negotiation.cc:407] Connection header received
0806 21:16:22.291610 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:22.291612 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:22.291654 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:22.291715 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:16:22.292065 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:22.292779 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:22.293644 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:22.293973 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:22.294073 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:22.294312 (+ 239us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:16:22.294392 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:16:22.294463 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:22.294594 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 21:16:22.294651 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 21:16:22.295469 19597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:22.290652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48718 (local address 127.16.178.62:42067)
0806 21:16:22.290988 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:22.290992 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:22.291107 (+ 115us) server_negotiation.cc:407] Connection header received
0806 21:16:22.291263 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:22.291265 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:22.291393 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:22.291501 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:16:22.291803 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:22.292563 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:22.293688 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:22.294034 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:22.294114 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:22.294520 (+ 406us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:16:22.294904 (+ 384us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:16:22.294989 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:22.295255 (+ 266us) server_negotiation.cc:299] Negotiation successful
0806 21:16:22.295322 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":71,"threads_started":1}
I20260806 21:16:22.298427 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 19046
I20260806 21:16:22.316322 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 19167
I20260806 21:16:22.331282 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 19272
2026-08-06T21:16:22Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3297 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 21:16:22 dist-test-slave-lgr5 krb5kdc[19603](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 21:16:22 dist-test-slave-lgr5 krb5kdc[19603](info): set up 2 sockets
Aug 06 21:16:22 dist-test-slave-lgr5 krb5kdc[19603](info): commencing operation
krb5kdc: starting...
W20260806 21:16:23.481413 17096 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/krb5kdc/test-user.keytab.
Aug 06 21:16:23 dist-test-slave-lgr5 krb5kdc[19603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T21:16:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T21:16:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.178.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
Entry for principal kudu/127.16.178.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab.
I20260806 21:16:23.609465 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:46469
--webserver_interface=127.16.178.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:36879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:46469,127.16.178.61:40205,127.16.178.60:35249
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 21:16:23.724623 19621 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 21:16:23.724921 19621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:16:23.724958 19621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:16:23.725021 19621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:16:23.729140 19621 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:16:23.729228 19621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:16:23.729255 19621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:16:23.729275 19621 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:16:23.729295 19621 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:16:23.735143 19621 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:36879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:46469,127.16.178.61:40205,127.16.178.60:35249
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.62.keytab
--principal=kudu/127.16.178.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.62:46469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.178.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19621
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:16:23.736335 19621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:16:23.737361 19621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:16:23.745558 19632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:16:23.745589 19630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:16:23.745584 19628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:16:23.745790 19621 server_base.cc:1034] running on GCE node
I20260806 21:16:23.746429 19621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:16:23.747530 19621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:16:23.748683 19621 hybrid_clock.cc:648] HybridClock initialized: now 1786050983748668 us; error 35 us; skew 500 ppm
Aug 06 21:16:23 dist-test-slave-lgr5 krb5kdc[19603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050983, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:16:23.751349 19621 init.cc:377] Logged in from keytab as kudu/127.16.178.62@KRBTEST.COM (short username kudu)
I20260806 21:16:23.752275 19621 webserver.cc:460] Webserver started at http://127.16.178.62:35393/ using document root <none> and password file <none>
I20260806 21:16:23.752822 19621 fs_manager.cc:359] Metadata directory not provided
I20260806 21:16:23.752887 19621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:16:23.753082 19621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:16:23.755157 19621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-0/data/instance:
uuid: "efb0fb1afff74bbf86828a74fe217e2e"
format_stamp: "Formatted at 2026-08-06 21:16:23 on dist-test-slave-lgr5"
server_key: "89e9ee18e3ba7b3e676207425205dd05"
server_key_iv: "f0d4b9d22fcfee53342b3f197acef332"
server_key_version: "encryptionkey@0"
I20260806 21:16:23.755729 19621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-0/wal/instance:
uuid: "efb0fb1afff74bbf86828a74fe217e2e"
format_stamp: "Formatted at 2026-08-06 21:16:23 on dist-test-slave-lgr5"
server_key: "89e9ee18e3ba7b3e676207425205dd05"
server_key_iv: "f0d4b9d22fcfee53342b3f197acef332"
server_key_version: "encryptionkey@0"
I20260806 21:16:23.759711 19621 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 21:16:23.762247 19645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:16:23.763463 19621 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 21:16:23.763604 19621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-0/wal
uuid: "efb0fb1afff74bbf86828a74fe217e2e"
format_stamp: "Formatted at 2026-08-06 21:16:23 on dist-test-slave-lgr5"
server_key: "89e9ee18e3ba7b3e676207425205dd05"
server_key_iv: "f0d4b9d22fcfee53342b3f197acef332"
server_key_version: "encryptionkey@0"
I20260806 21:16:23.763710 19621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:16:23.778069 19621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:16:23.781193 19621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:16:23.781388 19621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:16:23.788262 19621 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.62:46469
I20260806 21:16:23.788522 19715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.62:46469 every 8 connection(s)
I20260806 21:16:23.789294 19621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-0/data/info.pb
I20260806 21:16:23.792307 19717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:16:23.796324 19717 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.178.62" port: 46469 } has no permanent_uuid. Determining permanent_uuid...
I20260806 21:16:23.797025 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 19621
I20260806 21:16:23.797139 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-0/wal/instance
I20260806 21:16:23.797407 17096 external_mini_cluster.cc:1428] Setting key a3c3c432c99051144d482d68782ff72f
Aug 06 21:16:23 dist-test-slave-lgr5 krb5kdc[19603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050983, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.178.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.61@KRBTEST.COM" created.
I20260806 21:16:23.812810 19720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:23.798726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58190 (local address 127.16.178.62:46469)
0806 21:16:23.799161 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:23.799181 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:23.799584 (+ 403us) server_negotiation.cc:407] Connection header received
0806 21:16:23.800695 (+ 1111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:23.800700 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:23.800887 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:23.801054 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:23.801847 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:23.803175 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:23.804264 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:23.804863 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:23.805034 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:23.807665 (+ 2631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:23.807672 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:23.807687 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:16:23.807714 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:23.809581 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:23.810059 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:23.810064 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:23.810065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:23.810168 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:23.810517 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:23.810521 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:23.810522 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:23.811031 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:23.811162 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:23.811584 (+ 422us) server_negotiation.cc:299] Negotiation successful
0806 21:16:23.811938 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":116,"server-negotiator.queue_time_us":294,"thread_start_us":125,"threads_started":1}
I20260806 21:16:23.812814 19722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:23.798727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:46469 (local address 127.0.0.1:58190)
0806 21:16:23.799285 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:23.799321 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 21:16:23.800382 (+ 1061us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:16:23.801072 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:16:23.801075 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:16:23.801403 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:16:23.801682 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:16:23.801696 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:16:23.803321 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:16:23.803324 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:16:23.804128 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:16:23.804141 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:16:23.805026 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:16:23.805029 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:16:23.805128 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:23.805663 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:16:23.805690 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:16:23.807474 (+ 1784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:16:23.809715 (+ 2241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:16:23.809729 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:16:23.809737 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:16:23.809968 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:16:23.810277 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:16:23.810280 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:16:23.810281 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:16:23.810422 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:16:23.811154 (+ 732us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:16:23.811162 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:16:23.811325 (+ 163us) client_negotiation.cc:769] Sending connection context
0806 21:16:23.811481 (+ 156us) client_negotiation.cc:240] Negotiation successful
0806 21:16:23.811846 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"mutex_wait_us":47,"spinlock_wait_cycles":16256,"thread_start_us":153,"threads_started":1}
I20260806 21:16:23.815148 19717 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.178.61" port: 40205 } has no permanent_uuid. Determining permanent_uuid...
I20260806 21:16:23.815950 19722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:23.815627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.61:40205 (local address 127.0.0.1:57184)
0806 21:16:23.815776 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:23.815869 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.61:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 21:16:23.816195 19649 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.178.61:40205: connect: Connection refused (error 111)
I20260806 21:16:23.817081 19722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:23.816832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.61:40205 (local address 127.0.0.1:57188)
0806 21:16:23.816959 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:23.817015 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.61:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 21:16:23.817241 19717 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.178.61:40205: Network error: Client connection negotiation failed: client connection to 127.16.178.61:40205: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.61.keytab.
Entry for principal kudu/127.16.178.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.61.keytab.
I20260806 21:16:23.827108 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.61:40205
--webserver_interface=127.16.178.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.61.keytab
--principal=kudu/127.16.178.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:36879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:46469,127.16.178.61:40205,127.16.178.60:35249
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 21:16:23.866518 19717 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.178.61" port: 40205 } attempt: 1
I20260806 21:16:23.867540 19722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:23.867088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.61:40205 (local address 127.0.0.1:57194)
0806 21:16:23.867334 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:23.867428 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.61:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260806 21:16:23.868458 19722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:23.868180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.61:40205 (local address 127.0.0.1:57208)
0806 21:16:23.868315 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:23.868380 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.61:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 21:16:23.868677 19717 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.178.61:40205: Network error: Client connection negotiation failed: client connection to 127.16.178.61:40205: connect: Connection refused (error 111)
I20260806 21:16:23.936880 19717 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.178.61" port: 40205 } attempt: 2
I20260806 21:16:23.937805 19722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:23.937437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.61:40205 (local address 127.0.0.1:57224)
0806 21:16:23.937627 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:23.937706 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.61:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260806 21:16:23.938592 19722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:23.938344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.61:40205 (local address 127.0.0.1:57226)
0806 21:16:23.938449 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:23.938521 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.61:40205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 21:16:23.938732 19717 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.178.61:40205: Network error: Client connection negotiation failed: client connection to 127.16.178.61:40205: connect: Connection refused (error 111)
W20260806 21:16:23.946353 19728 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 21:16:23.946626 19728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:16:23.946648 19728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:16:23.946691 19728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 21:16:23.950697 19728 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:16:23.950754 19728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:16:23.950773 19728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:16:23.950786 19728 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:16:23.950800 19728 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:16:23.956178 19728 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:36879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:46469,127.16.178.61:40205,127.16.178.60:35249
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.61.keytab
--principal=kudu/127.16.178.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.61:40205
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.16.178.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19728
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:16:23.957213 19728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:16:23.958204 19728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:16:23.966711 19737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:16:23.966773 19738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:16:23.966773 19740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:16:23.966859 19728 server_base.cc:1034] running on GCE node
I20260806 21:16:23.967509 19728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:16:23.968693 19728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:16:23.969856 19728 hybrid_clock.cc:648] HybridClock initialized: now 1786050983969837 us; error 33 us; skew 500 ppm
Aug 06 21:16:23 dist-test-slave-lgr5 krb5kdc[19603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050983, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:16:23.973126 19728 init.cc:377] Logged in from keytab as kudu/127.16.178.61@KRBTEST.COM (short username kudu)
I20260806 21:16:23.974330 19728 webserver.cc:460] Webserver started at http://127.16.178.61:39707/ using document root <none> and password file <none>
I20260806 21:16:23.975062 19728 fs_manager.cc:359] Metadata directory not provided
I20260806 21:16:23.975158 19728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:16:23.975412 19728 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:16:23.978106 19728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-1/data/instance:
uuid: "a6e88d59cca4401792386acd201f9194"
format_stamp: "Formatted at 2026-08-06 21:16:23 on dist-test-slave-lgr5"
server_key: "8ef7a61278cc8da6d10adfaf50d47584"
server_key_iv: "5184fa5943bc3e4d0c1879480a0d2a26"
server_key_version: "encryptionkey@0"
I20260806 21:16:23.978817 19728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-1/wal/instance:
uuid: "a6e88d59cca4401792386acd201f9194"
format_stamp: "Formatted at 2026-08-06 21:16:23 on dist-test-slave-lgr5"
server_key: "8ef7a61278cc8da6d10adfaf50d47584"
server_key_iv: "5184fa5943bc3e4d0c1879480a0d2a26"
server_key_version: "encryptionkey@0"
I20260806 21:16:23.983104 19728 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 21:16:23.985918 19753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:16:23.987293 19728 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 21:16:23.987421 19728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-1/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-1/wal
uuid: "a6e88d59cca4401792386acd201f9194"
format_stamp: "Formatted at 2026-08-06 21:16:23 on dist-test-slave-lgr5"
server_key: "8ef7a61278cc8da6d10adfaf50d47584"
server_key_iv: "5184fa5943bc3e4d0c1879480a0d2a26"
server_key_version: "encryptionkey@0"
I20260806 21:16:23.987527 19728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:16:24.002547 19728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:16:24.005942 19728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:16:24.006138 19728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:16:24.013466 19728 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.61:40205
I20260806 21:16:24.013782 19831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.61:40205 every 8 connection(s)
I20260806 21:16:24.014523 19728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-1/data/info.pb
I20260806 21:16:24.017465 19833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:16:24.021493 19833 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.178.62" port: 46469 } has no permanent_uuid. Determining permanent_uuid...
I20260806 21:16:24.024689 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 19728
I20260806 21:16:24.024811 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-1/wal/instance
I20260806 21:16:24.025090 17096 external_mini_cluster.cc:1428] Setting key a4dd8c3852e6a78cfb20f5857afe5fae
I20260806 21:16:24.029875 19717 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.178.61" port: 40205 } attempt: 3
Aug 06 21:16:24 dist-test-slave-lgr5 krb5kdc[19603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050983, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.61@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.178.60@KRBTEST.COM; defaulting to no policy
I20260806 21:16:24.037179 19720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:24.023424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58196 (local address 127.16.178.62:46469)
0806 21:16:24.023557 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:24.023561 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:24.024377 (+ 816us) server_negotiation.cc:407] Connection header received
0806 21:16:24.025371 (+ 994us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:24.025374 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:24.025441 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:24.025533 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:24.026733 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:24.027350 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:24.028858 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:24.029215 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:24.029352 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:24.032354 (+ 3002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:24.032358 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:24.032361 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:16:24.032388 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:24.035166 (+ 2778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:24.035650 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:24.035654 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:24.035655 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:24.035728 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:24.036051 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:24.036054 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:24.036056 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:24.036267 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:24.036357 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:24.036884 (+ 527us) server_negotiation.cc:299] Negotiation successful
0806 21:16:24.037011 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.178.60@KRBTEST.COM" created.
Aug 06 21:16:24 dist-test-slave-lgr5 krb5kdc[19603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050983, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for kudu/127.16.178.61@KRBTEST.COM
I20260806 21:16:24.038120 19835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:24.023732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:46469 (local address 127.0.0.1:58196)
0806 21:16:24.024222 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:24.024269 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 21:16:24.025130 (+ 861us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:16:24.025692 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:16:24.025699 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:16:24.026052 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:16:24.026572 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:16:24.026588 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:16:24.027519 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:16:24.027523 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:16:24.028715 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:16:24.028727 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:16:24.029385 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:16:24.029389 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:16:24.029556 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:24.030224 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:16:24.030241 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:16:24.032207 (+ 1966us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:16:24.035272 (+ 3065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:16:24.035279 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:16:24.035295 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:16:24.035546 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:16:24.035806 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:16:24.035809 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:16:24.035811 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:16:24.035953 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:16:24.036367 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:16:24.036374 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:16:24.036630 (+ 256us) client_negotiation.cc:769] Sending connection context
0806 21:16:24.036793 (+ 163us) client_negotiation.cc:240] Negotiation successful
0806 21:16:24.037133 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":126,"threads_started":1}
I20260806 21:16:24.040012 19833 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.178.61" port: 40205 } has no permanent_uuid. Determining permanent_uuid...
I20260806 21:16:24.042119 19722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:24.030399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.61:40205 (local address 127.0.0.1:57236)
0806 21:16:24.030611 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:24.030629 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 21:16:24.030714 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:16:24.031363 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:16:24.031366 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:16:24.031543 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:16:24.031644 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:16:24.031651 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:16:24.032713 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:16:24.032716 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:16:24.033264 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:16:24.033273 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:16:24.034080 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:16:24.034085 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:16:24.034214 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:24.034751 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:16:24.034778 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:16:24.038384 (+ 3606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:16:24.040431 (+ 2047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:16:24.040434 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:16:24.040436 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:16:24.040665 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:16:24.041014 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:16:24.041018 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:16:24.041020 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:16:24.041123 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:16:24.041619 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:16:24.041622 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:16:24.041725 (+ 103us) client_negotiation.cc:769] Sending connection context
0806 21:16:24.041829 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 21:16:24.041969 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131}
I20260806 21:16:24.042403 19839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:24.030498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57236 (local address 127.16.178.61:40205)
0806 21:16:24.030981 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:24.030992 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:24.031013 (+ 21us) server_negotiation.cc:407] Connection header received
0806 21:16:24.031048 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:24.031052 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:24.031206 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:24.031354 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:24.031744 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:24.032579 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:24.033372 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:24.033929 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:24.034040 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:24.038499 (+ 4459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:24.038504 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:24.038514 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:16:24.038535 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:24.040300 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:24.040796 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:24.040802 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:24.040804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:24.040889 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:24.041210 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:24.041213 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:24.041215 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:24.041506 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:24.041639 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:24.041921 (+ 282us) server_negotiation.cc:299] Negotiation successful
0806 21:16:24.042064 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":163,"threads_started":1}
I20260806 21:16:24.044313 19717 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.178.60" port: 35249 } has no permanent_uuid. Determining permanent_uuid...
I20260806 21:16:24.044997 19722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:24.044760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:35249 (local address 127.0.0.1:59558)
0806 21:16:24.044881 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:24.044936 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.60:35249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 21:16:24.045665 19722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:24.045421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:35249 (local address 127.0.0.1:59566)
0806 21:16:24.045552 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:24.045598 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.60:35249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260806 21:16:24.045848 19717 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.178.60:35249: Network error: Client connection negotiation failed: client connection to 127.16.178.60:35249: connect: Connection refused (error 111)
Aug 06 21:16:24 dist-test-slave-lgr5 krb5kdc[19603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050983, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.61@KRBTEST.COM for kudu/127.16.178.61@KRBTEST.COM
I20260806 21:16:24.050575 19835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:24.040550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.61:40205 (local address 127.0.0.1:57246)
0806 21:16:24.040724 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:24.040737 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 21:16:24.040813 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:16:24.041044 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:16:24.041046 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:16:24.041201 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:16:24.041312 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:16:24.041318 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:16:24.042478 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:16:24.043297 (+ 819us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:16:24.044142 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:16:24.044155 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:16:24.045369 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:16:24.045372 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:16:24.045468 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:24.045992 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:16:24.046008 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:16:24.047355 (+ 1347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:16:24.049155 (+ 1800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:16:24.049158 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:16:24.049160 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:16:24.049377 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:16:24.049679 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:16:24.049681 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:16:24.049683 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:16:24.049761 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:16:24.050135 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:16:24.050138 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:16:24.050227 (+ 89us) client_negotiation.cc:769] Sending connection context
0806 21:16:24.050337 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 21:16:24.050453 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105}
I20260806 21:16:24.050683 19841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:24.040581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57246 (local address 127.16.178.61:40205)
0806 21:16:24.040848 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:24.040864 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:24.040877 (+ 13us) server_negotiation.cc:407] Connection header received
0806 21:16:24.040906 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:24.040909 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:24.040946 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:24.041033 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:24.041429 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:24.042100 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:24.044275 (+ 2175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:24.045151 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:24.045258 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:24.047592 (+ 2334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:24.047595 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:24.047597 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:16:24.047620 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:24.049043 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:24.049479 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:24.049482 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:24.049484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:24.049581 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:24.049837 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:24.049839 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:24.049840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:24.050039 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:24.050117 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:24.050384 (+ 267us) server_negotiation.cc:299] Negotiation successful
0806 21:16:24.050515 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":16,"server-negotiator.queue_time_us":213,"thread_start_us":104,"threads_started":1}
I20260806 21:16:24.051374 19833 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.178.60" port: 35249 } has no permanent_uuid. Determining permanent_uuid...
I20260806 21:16:24.052178 19835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:24.051809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:35249 (local address 127.0.0.1:59576)
0806 21:16:24.051955 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:24.052066 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.60:35249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260806 21:16:24.052409 19755 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.178.60:35249: connect: Connection refused (error 111)
I20260806 21:16:24.053318 19835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:24.053095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:35249 (local address 127.0.0.1:59588)
0806 21:16:24.053202 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:24.053256 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.60:35249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260806 21:16:24.053459 19833 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.178.60:35249: Network error: Client connection negotiation failed: client connection to 127.16.178.60:35249: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.178.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.60.keytab.
Entry for principal kudu/127.16.178.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.60.keytab.
I20260806 21:16:24.055044 17096 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
/tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.60:35249
--webserver_interface=127.16.178.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.60.keytab
--principal=kudu/127.16.178.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.178.20:36879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.178.62:46469,127.16.178.61:40205,127.16.178.60:35249
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 21:16:24.077010 19717 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.178.60" port: 35249 } attempt: 1
I20260806 21:16:24.077920 19722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:24.077547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:35249 (local address 127.0.0.1:59594)
0806 21:16:24.077735 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:24.077818 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.60:35249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260806 21:16:24.078634 19722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:24.078415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:35249 (local address 127.0.0.1:59606)
0806 21:16:24.078513 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:24.078566 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.60:35249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 21:16:24.078763 19717 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.178.60:35249: Network error: Client connection negotiation failed: client connection to 127.16.178.60:35249: connect: Connection refused (error 111)
I20260806 21:16:24.102738 19833 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.178.60" port: 35249 } attempt: 1
I20260806 21:16:24.103626 19835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:24.103316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:35249 (local address 127.0.0.1:59616)
0806 21:16:24.103476 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:24.103543 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.60:35249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260806 21:16:24.104413 19835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:24.104212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:35249 (local address 127.0.0.1:59630)
0806 21:16:24.104304 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:24.104351 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.60:35249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 21:16:24.104619 19833 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.178.60:35249: Network error: Client connection negotiation failed: client connection to 127.16.178.60:35249: connect: Connection refused (error 111)
I20260806 21:16:24.153931 19717 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.178.60" port: 35249 } attempt: 2
I20260806 21:16:24.154881 19722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:24.154522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:35249 (local address 127.0.0.1:59638)
0806 21:16:24.154701 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:24.154779 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.60:35249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260806 21:16:24.155632 19722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:24.155404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:35249 (local address 127.0.0.1:59650)
0806 21:16:24.155510 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:24.155564 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.60:35249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 21:16:24.155792 19717 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.178.60:35249: Network error: Client connection negotiation failed: client connection to 127.16.178.60:35249: connect: Connection refused (error 111)
W20260806 21:16:24.169701 19844 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 21:16:24.170030 19844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 21:16:24.170055 19844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 21:16:24.170114 19844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 21:16:24.172760 19833 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.178.60" port: 35249 } attempt: 2
I20260806 21:16:24.173720 19835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:24.173364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:35249 (local address 127.0.0.1:59662)
0806 21:16:24.173562 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:24.173634 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.60:35249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260806 21:16:24.174374 19835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:24.174184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:35249 (local address 127.0.0.1:59676)
0806 21:16:24.174264 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:24.174310 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.178.60:35249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 21:16:24.174482 19833 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.178.60:35249: Network error: Client connection negotiation failed: client connection to 127.16.178.60:35249: connect: Connection refused (error 111)
W20260806 21:16:24.174577 19844 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 21:16:24.174638 19844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 21:16:24.174667 19844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 21:16:24.174690 19844 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 21:16:24.174710 19844 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 21:16:24.180284 19844 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.178.20:36879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.16.178.62:46469,127.16.178.61:40205,127.16.178.60:35249
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/krb5kdc/kudu_127.16.178.60.keytab
--principal=kudu/127.16.178.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.178.60:35249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.16.178.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 20:33:17 UTC on ccycloud.quasar-matgds.root.comops.site
build id 4571390
I20260806 21:16:24.181384 19844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 21:16:24.182422 19844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 21:16:24.191267 19852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:16:24.191314 19854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 21:16:24.191275 19851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 21:16:24.191627 19844 server_base.cc:1034] running on GCE node
I20260806 21:16:24.192291 19844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 21:16:24.193464 19844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 21:16:24.194625 19844 hybrid_clock.cc:648] HybridClock initialized: now 1786050984194608 us; error 30 us; skew 500 ppm
Aug 06 21:16:24 dist-test-slave-lgr5 krb5kdc[19603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050984, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 21:16:24.197413 19844 init.cc:377] Logged in from keytab as kudu/127.16.178.60@KRBTEST.COM (short username kudu)
I20260806 21:16:24.198395 19844 webserver.cc:460] Webserver started at http://127.16.178.60:38397/ using document root <none> and password file <none>
I20260806 21:16:24.198946 19844 fs_manager.cc:359] Metadata directory not provided
I20260806 21:16:24.199009 19844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 21:16:24.199199 19844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 21:16:24.201565 19844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-2/data/instance:
uuid: "e4b6f9d7614a489599e60be14993a722"
format_stamp: "Formatted at 2026-08-06 21:16:24 on dist-test-slave-lgr5"
server_key: "2fa979bd2b653a9a2dfb978a29001cb4"
server_key_iv: "1afd08e18e90d8d116e8616c3f2c78d3"
server_key_version: "encryptionkey@0"
I20260806 21:16:24.202173 19844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-2/wal/instance:
uuid: "e4b6f9d7614a489599e60be14993a722"
format_stamp: "Formatted at 2026-08-06 21:16:24 on dist-test-slave-lgr5"
server_key: "2fa979bd2b653a9a2dfb978a29001cb4"
server_key_iv: "1afd08e18e90d8d116e8616c3f2c78d3"
server_key_version: "encryptionkey@0"
I20260806 21:16:24.206245 19844 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 21:16:24.208997 19862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 21:16:24.210435 19844 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 21:16:24.210616 19844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-2/data,/tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-2/wal
uuid: "e4b6f9d7614a489599e60be14993a722"
format_stamp: "Formatted at 2026-08-06 21:16:24 on dist-test-slave-lgr5"
server_key: "2fa979bd2b653a9a2dfb978a29001cb4"
server_key_iv: "1afd08e18e90d8d116e8616c3f2c78d3"
server_key_version: "encryptionkey@0"
I20260806 21:16:24.210749 19844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 21:16:24.231350 19844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 21:16:24.234638 19844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 21:16:24.234853 19844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 21:16:24.241894 19844 rpc_server.cc:307] RPC server started. Bound to: 127.16.178.60:35249
I20260806 21:16:24.242205 19939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.178.60:35249 every 8 connection(s)
I20260806 21:16:24.242882 19844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-2/data/info.pb
I20260806 21:16:24.245718 19940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 21:16:24.249670 19940 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.178.62" port: 46469 } has no permanent_uuid. Determining permanent_uuid...
I20260806 21:16:24.252678 17096 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu as pid 19844
I20260806 21:16:24.252817 17096 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ1T3_n/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786050686797304-17096-0/minicluster-data/master-2/wal/instance
I20260806 21:16:24.253173 17096 external_mini_cluster.cc:1428] Setting key 05835397014f10b007d1bda0032a369e
I20260806 21:16:24.254953 19717 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.178.60" port: 35249 } attempt: 3
Aug 06 21:16:24 dist-test-slave-lgr5 krb5kdc[19603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050984, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.60@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
Aug 06 21:16:24 dist-test-slave-lgr5 krb5kdc[19603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050983, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.62@KRBTEST.COM for kudu/127.16.178.60@KRBTEST.COM
Aug 06 21:16:24 dist-test-slave-lgr5 krb5kdc[19603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.178.60@KRBTEST.COM
I20260806 21:16:24.265300 19720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:24.251580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58208 (local address 127.16.178.62:46469)
0806 21:16:24.251710 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:24.251714 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:24.252638 (+ 924us) server_negotiation.cc:407] Connection header received
0806 21:16:24.253883 (+ 1245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:24.253887 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:24.253955 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:24.254053 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:24.255781 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:24.256440 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:24.257607 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:24.257904 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:24.258006 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:24.260814 (+ 2808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:24.260819 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:24.260822 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:16:24.260848 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:24.262548 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:24.263453 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:24.263457 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:24.263459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:24.263534 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:24.263975 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:24.263978 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:24.263980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:24.264303 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:24.264391 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:24.264962 (+ 571us) server_negotiation.cc:299] Negotiation successful
0806 21:16:24.265116 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":46}
I20260806 21:16:24.265712 19722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:24.255431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:35249 (local address 127.0.0.1:59708)
0806 21:16:24.255558 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:24.255574 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:16:24.255658 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:16:24.256048 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:16:24.256051 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:16:24.256255 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:16:24.256344 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:16:24.256350 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:16:24.257543 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:16:24.257546 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:16:24.258327 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:16:24.258336 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:16:24.259251 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:16:24.259255 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:16:24.259340 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:24.259788 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:16:24.259803 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:16:24.261096 (+ 1293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:16:24.263395 (+ 2299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:16:24.263398 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:16:24.263400 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:16:24.263640 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:16:24.263942 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:16:24.263945 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:16:24.263947 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:16:24.264036 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:16:24.264820 (+ 784us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:16:24.264823 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:16:24.264922 (+ 99us) client_negotiation.cc:769] Sending connection context
0806 21:16:24.265020 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 21:16:24.265569 (+ 549us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":447}
I20260806 21:16:24.265657 19833 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.178.60" port: 35249 } attempt: 3
I20260806 21:16:24.266625 19942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:24.251927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.62:46469 (local address 127.0.0.1:58208)
0806 21:16:24.252482 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:24.252528 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 21:16:24.253640 (+ 1112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:16:24.254291 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:16:24.254301 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:16:24.254784 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:16:24.255394 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:16:24.255685 (+ 291us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:16:24.256540 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:16:24.256544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:16:24.257457 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:16:24.257470 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:16:24.258063 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:16:24.258067 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:16:24.258278 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:24.258896 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:16:24.258915 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:16:24.260647 (+ 1732us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:16:24.262665 (+ 2018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:16:24.262677 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:16:24.262690 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:16:24.263317 (+ 627us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:16:24.263630 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:16:24.263633 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:16:24.263636 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:16:24.263848 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:16:24.264409 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:16:24.264417 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:16:24.264690 (+ 273us) client_negotiation.cc:769] Sending connection context
0806 21:16:24.264900 (+ 210us) client_negotiation.cc:240] Negotiation successful
0806 21:16:24.265247 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"mutex_wait_us":21,"spinlock_wait_cycles":4352,"thread_start_us":172,"threads_started":1}
I20260806 21:16:24.266625 19948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:24.255466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59708 (local address 127.16.178.60:35249)
0806 21:16:24.255807 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:24.255821 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:24.255835 (+ 14us) server_negotiation.cc:407] Connection header received
0806 21:16:24.255868 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:24.255870 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:24.255906 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:24.256009 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:24.256505 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:24.257394 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:24.258428 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:24.259110 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:24.259543 (+ 433us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:24.261241 (+ 1698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:24.261249 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:24.261264 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:16:24.261288 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:24.263292 (+ 2004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:24.263753 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:24.263760 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:24.263762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:24.263856 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:24.264173 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:24.264177 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:24.264178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:24.264711 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:24.264840 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:24.265202 (+ 362us) server_negotiation.cc:299] Negotiation successful
0806 21:16:24.265328 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":182,"server-negotiator.queue_time_us":270,"spinlock_wait_cycles":21504,"thread_start_us":134,"threads_started":1}
I20260806 21:16:24.266938 19946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:24.254120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59692 (local address 127.16.178.60:35249)
0806 21:16:24.254402 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:24.254410 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:24.254516 (+ 106us) server_negotiation.cc:407] Connection header received
0806 21:16:24.254658 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:24.254663 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:24.254823 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:24.254951 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:24.256142 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:24.257393 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:24.258453 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:24.259110 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:24.259213 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:24.262043 (+ 2830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:24.262047 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:24.262049 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:16:24.262075 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:24.264312 (+ 2237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:24.264821 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:24.264824 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:24.264826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:24.264934 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:24.265250 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:24.265254 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:24.265255 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:24.265729 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:24.266216 (+ 487us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:24.266596 (+ 380us) server_negotiation.cc:299] Negotiation successful
0806 21:16:24.266709 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":68,"server-negotiator.queue_time_us":194,"spinlock_wait_cycles":896,"thread_start_us":111,"threads_started":1}
I20260806 21:16:24.268208 19940 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.178.61" port: 40205 } has no permanent_uuid. Determining permanent_uuid...
I20260806 21:16:24.271435 19717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efb0fb1afff74bbf86828a74fe217e2e: Bootstrap starting.
Aug 06 21:16:24 dist-test-slave-lgr5 krb5kdc[19603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050983, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.61@KRBTEST.COM for kudu/127.16.178.60@KRBTEST.COM
Aug 06 21:16:24 dist-test-slave-lgr5 krb5kdc[19603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050984, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.60@KRBTEST.COM for kudu/127.16.178.61@KRBTEST.COM
Aug 06 21:16:24 dist-test-slave-lgr5 krb5kdc[19603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.178.62@KRBTEST.COM
I20260806 21:16:24.275835 19717 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P efb0fb1afff74bbf86828a74fe217e2e: Neither blocks nor log segments found. Creating new log.
I20260806 21:16:24.277014 19717 log.cc:824] T 00000000000000000000000000000000 P efb0fb1afff74bbf86828a74fe217e2e: Log is configured to *not* fsync() on all Append() calls
I20260806 21:16:24.277406 19835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:24.266122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:35249 (local address 127.0.0.1:59716)
0806 21:16:24.266259 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:24.266275 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 21:16:24.266359 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:16:24.267435 (+ 1076us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:16:24.267438 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:16:24.267629 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:16:24.267749 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:16:24.267757 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:16:24.269266 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:16:24.269270 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:16:24.270021 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:16:24.270031 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:16:24.271498 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:16:24.271502 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:16:24.271594 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:24.272157 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:16:24.272173 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:16:24.273726 (+ 1553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:16:24.275617 (+ 1891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:16:24.275622 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:16:24.275624 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:16:24.275899 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:16:24.276291 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:16:24.276294 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:16:24.276296 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:16:24.276386 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:16:24.276861 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:16:24.276864 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:16:24.276982 (+ 118us) client_negotiation.cc:769] Sending connection context
0806 21:16:24.277084 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 21:16:24.277258 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 21:16:24.278092 19946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:24.266156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59716 (local address 127.16.178.60:35249)
0806 21:16:24.267227 (+ 1071us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:24.267231 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:24.267244 (+ 13us) server_negotiation.cc:407] Connection header received
0806 21:16:24.267281 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:24.267283 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:24.267325 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:24.267398 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:24.267879 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:24.269156 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:24.271011 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:24.271366 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:24.271825 (+ 459us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:24.273892 (+ 2067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:24.273896 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:24.273899 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:16:24.273925 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:24.275454 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:24.276053 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:24.276056 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:24.276058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:24.276182 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:24.276481 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:24.276484 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:24.276486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:24.276748 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:24.276851 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:24.277792 (+ 941us) server_negotiation.cc:299] Negotiation successful
0806 21:16:24.277920 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":1009,"thread_start_us":113,"threads_started":1}
I20260806 21:16:24.279891 19717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efb0fb1afff74bbf86828a74fe217e2e: No bootstrap required, opened a new log
I20260806 21:16:24.280036 19942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:24.268695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.61:40205 (local address 127.0.0.1:57252)
0806 21:16:24.269039 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:24.269053 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:16:24.269122 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:16:24.269438 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:16:24.269441 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:16:24.269625 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:16:24.269731 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:16:24.269738 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:16:24.270972 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:16:24.270975 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:16:24.271729 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:16:24.271739 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:16:24.272769 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:16:24.272772 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:16:24.272878 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:24.273349 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:16:24.273364 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:16:24.276055 (+ 2691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:16:24.277998 (+ 1943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:16:24.278002 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:16:24.278004 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:16:24.278485 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:16:24.278823 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:16:24.278826 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:16:24.278828 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:16:24.278923 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:16:24.279487 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:16:24.279491 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:16:24.279601 (+ 110us) client_negotiation.cc:769] Sending connection context
0806 21:16:24.279712 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 21:16:24.279861 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":24}
I20260806 21:16:24.280134 19841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:24.268931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57252 (local address 127.16.178.61:40205)
0806 21:16:24.269078 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:24.269082 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:24.269097 (+ 15us) server_negotiation.cc:407] Connection header received
0806 21:16:24.269232 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:24.269235 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:24.269285 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:24.269333 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:24.270304 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:24.270862 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:24.272183 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:24.272527 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:24.272607 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:24.276215 (+ 3608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:24.276219 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:24.276222 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:16:24.276250 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:24.277865 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:24.278604 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:24.278608 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:24.278609 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:24.278710 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:24.279058 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:24.279062 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:24.279064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:24.279371 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:24.279484 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:24.279790 (+ 306us) server_negotiation.cc:299] Negotiation successful
0806 21:16:24.279934 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":843,"server-negotiator.queue_time_us":65}
I20260806 21:16:24.280887 19940 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.178.60" port: 35249 } has no permanent_uuid. Determining permanent_uuid...
I20260806 21:16:24.282162 19720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:24.270153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58218 (local address 127.16.178.62:46469)
0806 21:16:24.270278 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:24.270282 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:24.270294 (+ 12us) server_negotiation.cc:407] Connection header received
0806 21:16:24.270327 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:24.270330 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:24.270375 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:24.270452 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:24.271192 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:24.271841 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:24.272935 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:24.273251 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:24.273325 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:24.278366 (+ 5041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:24.278370 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:24.278373 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:16:24.278405 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:24.279957 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:24.280593 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:24.280596 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:24.280598 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:24.280702 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:24.281029 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:24.281032 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:24.281034 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:24.281262 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:24.281446 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:24.281833 (+ 387us) server_negotiation.cc:299] Negotiation successful
0806 21:16:24.281980 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 21:16:24.284787 19717 raft_consensus.cc:348] T 00000000000000000000000000000000 P efb0fb1afff74bbf86828a74fe217e2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efb0fb1afff74bbf86828a74fe217e2e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 46469 } } peers { permanent_uuid: "a6e88d59cca4401792386acd201f9194" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 40205 } } peers { permanent_uuid: "e4b6f9d7614a489599e60be14993a722" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 35249 } }
I20260806 21:16:24.285109 19717 raft_consensus.cc:729] T 00000000000000000000000000000000 P efb0fb1afff74bbf86828a74fe217e2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efb0fb1afff74bbf86828a74fe217e2e, State: Initialized, Role: FOLLOWER
I20260806 21:16:24.285522 19717 consensus_queue.cc:260] T 00000000000000000000000000000000 P efb0fb1afff74bbf86828a74fe217e2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efb0fb1afff74bbf86828a74fe217e2e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 46469 } } peers { permanent_uuid: "a6e88d59cca4401792386acd201f9194" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 40205 } } peers { permanent_uuid: "e4b6f9d7614a489599e60be14993a722" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 35249 } }
I20260806 21:16:24.286150 19953 sys_catalog.cc:455] T 00000000000000000000000000000000 P efb0fb1afff74bbf86828a74fe217e2e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efb0fb1afff74bbf86828a74fe217e2e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 46469 } } peers { permanent_uuid: "a6e88d59cca4401792386acd201f9194" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 40205 } } peers { permanent_uuid: "e4b6f9d7614a489599e60be14993a722" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 35249 } } }
I20260806 21:16:24.286409 19953 sys_catalog.cc:458] T 00000000000000000000000000000000 P efb0fb1afff74bbf86828a74fe217e2e [sys.catalog]: This master's current role is: FOLLOWER
Aug 06 21:16:24 dist-test-slave-lgr5 krb5kdc[19603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050984, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.178.60@KRBTEST.COM for kudu/127.16.178.60@KRBTEST.COM
I20260806 21:16:24.287211 19717 sys_catalog.cc:564] T 00000000000000000000000000000000 P efb0fb1afff74bbf86828a74fe217e2e [sys.catalog]: configured and running, proceeding with master startup.
Aug 06 21:16:24 dist-test-slave-lgr5 krb5kdc[19603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786050983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.178.61@KRBTEST.COM
I20260806 21:16:24.291322 19833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6e88d59cca4401792386acd201f9194: Bootstrap starting.
I20260806 21:16:24.291538 19946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:24.281622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59726 (local address 127.16.178.60:35249)
0806 21:16:24.281926 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:24.281928 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:24.281936 (+ 8us) server_negotiation.cc:407] Connection header received
0806 21:16:24.281956 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:24.281958 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:24.281989 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:24.282045 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:24.282634 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:24.283316 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:24.284412 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:24.284723 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:24.284826 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:24.287128 (+ 2302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:24.287132 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:24.287135 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:16:24.287162 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:24.288729 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:24.289258 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:24.289262 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:24.289264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:24.289361 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:24.289674 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:24.289678 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:24.289680 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:24.289947 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:24.290079 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:24.291276 (+ 1197us) server_negotiation.cc:299] Negotiation successful
0806 21:16:24.291399 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255}
I20260806 21:16:24.291668 19942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:24.281403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.178.60:35249 (local address 127.0.0.1:59726)
0806 21:16:24.281725 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0806 21:16:24.281739 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 21:16:24.281823 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 21:16:24.282087 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 21:16:24.282091 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 21:16:24.282400 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0806 21:16:24.282506 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:16:24.282513 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:16:24.283434 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:16:24.283437 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:16:24.284287 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 21:16:24.284300 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 21:16:24.284853 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 21:16:24.284857 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 21:16:24.284969 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:24.285466 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 21:16:24.285481 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 21:16:24.286995 (+ 1514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 21:16:24.288865 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:16:24.288869 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:16:24.288871 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:16:24.289149 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:16:24.289462 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 21:16:24.289464 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 21:16:24.289467 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 21:16:24.289568 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 21:16:24.290308 (+ 740us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 21:16:24.290311 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 21:16:24.290423 (+ 112us) client_negotiation.cc:769] Sending connection context
0806 21:16:24.290538 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 21:16:24.290691 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249}
I20260806 21:16:24.294543 19833 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a6e88d59cca4401792386acd201f9194: Neither blocks nor log segments found. Creating new log.
I20260806 21:16:24.294978 19841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:24.283908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57258 (local address 127.16.178.61:40205)
0806 21:16:24.284022 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:24.284027 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:24.284137 (+ 110us) server_negotiation.cc:407] Connection header received
0806 21:16:24.284240 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:24.284243 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:24.284293 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:24.284360 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:24.285040 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:24.286050 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:24.287108 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:24.287715 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:24.288103 (+ 388us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:24.290655 (+ 2552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:24.290659 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:24.290662 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:16:24.290694 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:24.292770 (+ 2076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:24.293346 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:24.293350 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:24.293352 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:24.293455 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:24.293798 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:24.293802 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:24.293803 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:24.294067 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:24.294289 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:24.294514 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 21:16:24.294652 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 21:16:24.295238 19940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4b6f9d7614a489599e60be14993a722: Bootstrap starting.
I20260806 21:16:24.295536 19833 log.cc:824] T 00000000000000000000000000000000 P a6e88d59cca4401792386acd201f9194: Log is configured to *not* fsync() on all Append() calls
I20260806 21:16:24.298229 19940 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e4b6f9d7614a489599e60be14993a722: Neither blocks nor log segments found. Creating new log.
I20260806 21:16:24.298933 17096 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 21:16:24.299144 19833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6e88d59cca4401792386acd201f9194: No bootstrap required, opened a new log
I20260806 21:16:24.299149 19940 log.cc:824] T 00000000000000000000000000000000 P e4b6f9d7614a489599e60be14993a722: Log is configured to *not* fsync() on all Append() calls
I20260806 21:16:24.300215 19682 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "efb0fb1afff74bbf86828a74fe217e2e"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58218
I20260806 21:16:24.300683 19682 raft_consensus.cc:482] T 00000000000000000000000000000000 P efb0fb1afff74bbf86828a74fe217e2e [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 21:16:24.300805 19682 raft_consensus.cc:3037] T 00000000000000000000000000000000 P efb0fb1afff74bbf86828a74fe217e2e [term 0 FOLLOWER]: Advancing to term 1
W20260806 21:16:24.301330 19969 catalog_manager.cc:1558] T 00000000000000000000000000000000 P efb0fb1afff74bbf86828a74fe217e2e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 21:16:24.301398 19969 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 21:16:24.302831 19682 raft_consensus.cc:504] T 00000000000000000000000000000000 P efb0fb1afff74bbf86828a74fe217e2e [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efb0fb1afff74bbf86828a74fe217e2e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 46469 } } peers { permanent_uuid: "a6e88d59cca4401792386acd201f9194" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 40205 } } peers { permanent_uuid: "e4b6f9d7614a489599e60be14993a722" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 35249 } }
I20260806 21:16:24.303126 19833 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6e88d59cca4401792386acd201f9194 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efb0fb1afff74bbf86828a74fe217e2e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 46469 } } peers { permanent_uuid: "a6e88d59cca4401792386acd201f9194" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 40205 } } peers { permanent_uuid: "e4b6f9d7614a489599e60be14993a722" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 35249 } }
I20260806 21:16:24.303344 19833 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6e88d59cca4401792386acd201f9194 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6e88d59cca4401792386acd201f9194, State: Initialized, Role: FOLLOWER
I20260806 21:16:24.303735 19833 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6e88d59cca4401792386acd201f9194 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efb0fb1afff74bbf86828a74fe217e2e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 46469 } } peers { permanent_uuid: "a6e88d59cca4401792386acd201f9194" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 40205 } } peers { permanent_uuid: "e4b6f9d7614a489599e60be14993a722" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 35249 } }
I20260806 21:16:24.305269 19833 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6e88d59cca4401792386acd201f9194 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:16:24.305769 19792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "efb0fb1afff74bbf86828a74fe217e2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a6e88d59cca4401792386acd201f9194"
I20260806 21:16:24.306058 19792 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6e88d59cca4401792386acd201f9194 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:16:24.307626 19792 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a6e88d59cca4401792386acd201f9194 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efb0fb1afff74bbf86828a74fe217e2e in term 1.
I20260806 21:16:24.308701 19649 leader_election.cc:302] T 00000000000000000000000000000000 P efb0fb1afff74bbf86828a74fe217e2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6e88d59cca4401792386acd201f9194, efb0fb1afff74bbf86828a74fe217e2e; no voters:
I20260806 21:16:24.309038 19953 raft_consensus.cc:2781] T 00000000000000000000000000000000 P efb0fb1afff74bbf86828a74fe217e2e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 21:16:24.309291 19953 raft_consensus.cc:686] T 00000000000000000000000000000000 P efb0fb1afff74bbf86828a74fe217e2e [term 1 LEADER]: Becoming Leader. State: Replica: efb0fb1afff74bbf86828a74fe217e2e, State: Running, Role: LEADER
I20260806 21:16:24.309310 19682 leader_election.cc:288] T 00000000000000000000000000000000 P efb0fb1afff74bbf86828a74fe217e2e [CANDIDATE]: Term 1 election: Requested vote from peers a6e88d59cca4401792386acd201f9194 (127.16.178.61:40205), e4b6f9d7614a489599e60be14993a722 (127.16.178.60:35249)
I20260806 21:16:24.309695 19953 consensus_queue.cc:237] T 00000000000000000000000000000000 P efb0fb1afff74bbf86828a74fe217e2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efb0fb1afff74bbf86828a74fe217e2e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 46469 } } peers { permanent_uuid: "a6e88d59cca4401792386acd201f9194" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 40205 } } peers { permanent_uuid: "e4b6f9d7614a489599e60be14993a722" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 35249 } }
W20260806 21:16:24.311324 19992 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a6e88d59cca4401792386acd201f9194: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 21:16:24.311412 19992 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 21:16:24.311802 19953 sys_catalog.cc:455] T 00000000000000000000000000000000 P efb0fb1afff74bbf86828a74fe217e2e [sys.catalog]: SysCatalogTable state changed. Reason: New leader efb0fb1afff74bbf86828a74fe217e2e. Latest consensus state: current_term: 1 leader_uuid: "efb0fb1afff74bbf86828a74fe217e2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efb0fb1afff74bbf86828a74fe217e2e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 46469 } } peers { permanent_uuid: "a6e88d59cca4401792386acd201f9194" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 40205 } } peers { permanent_uuid: "e4b6f9d7614a489599e60be14993a722" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 35249 } } }
I20260806 21:16:24.311932 19953 sys_catalog.cc:458] T 00000000000000000000000000000000 P efb0fb1afff74bbf86828a74fe217e2e [sys.catalog]: This master's current role is: LEADER
I20260806 21:16:24.312502 20004 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 21:16:24.313069 19940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4b6f9d7614a489599e60be14993a722: No bootstrap required, opened a new log
I20260806 21:16:24.314477 20004 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 21:16:24.309055 19621 master_runner.cc:186] Error getting master registration for 127.16.178.61:40205: OK, instance_id { permanent_uuid: "a6e88d59cca4401792386acd201f9194" instance_seqno: 1786050984011295 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 21:16:24.316571 19621 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 21:16:24.319119 19940 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4b6f9d7614a489599e60be14993a722 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efb0fb1afff74bbf86828a74fe217e2e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 46469 } } peers { permanent_uuid: "a6e88d59cca4401792386acd201f9194" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 40205 } } peers { permanent_uuid: "e4b6f9d7614a489599e60be14993a722" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 35249 } }
I20260806 21:16:24.319496 19940 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4b6f9d7614a489599e60be14993a722 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4b6f9d7614a489599e60be14993a722, State: Initialized, Role: FOLLOWER
I20260806 21:16:24.319912 19940 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4b6f9d7614a489599e60be14993a722 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efb0fb1afff74bbf86828a74fe217e2e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 46469 } } peers { permanent_uuid: "a6e88d59cca4401792386acd201f9194" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 40205 } } peers { permanent_uuid: "e4b6f9d7614a489599e60be14993a722" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 35249 } }
I20260806 21:16:24.320705 20010 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4b6f9d7614a489599e60be14993a722 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efb0fb1afff74bbf86828a74fe217e2e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 46469 } } peers { permanent_uuid: "a6e88d59cca4401792386acd201f9194" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 40205 } } peers { permanent_uuid: "e4b6f9d7614a489599e60be14993a722" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 35249 } } }
I20260806 21:16:24.321048 20010 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4b6f9d7614a489599e60be14993a722 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:16:24.321762 19940 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4b6f9d7614a489599e60be14993a722 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 21:16:24.321849 19901 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "efb0fb1afff74bbf86828a74fe217e2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e4b6f9d7614a489599e60be14993a722"
I20260806 21:16:24.322119 19901 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4b6f9d7614a489599e60be14993a722 [term 0 FOLLOWER]: Advancing to term 1
I20260806 21:16:24.319286 19728 master_runner.cc:186] Error getting master registration for 127.16.178.60:35249: OK, instance_id { permanent_uuid: "e4b6f9d7614a489599e60be14993a722" instance_seqno: 1786050984239928 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 21:16:24.322815 19728 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 21:16:24.323587 19901 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e4b6f9d7614a489599e60be14993a722 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efb0fb1afff74bbf86828a74fe217e2e in term 1.
I20260806 21:16:24.324384 19976 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6e88d59cca4401792386acd201f9194 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efb0fb1afff74bbf86828a74fe217e2e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 46469 } } peers { permanent_uuid: "a6e88d59cca4401792386acd201f9194" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 40205 } } peers { permanent_uuid: "e4b6f9d7614a489599e60be14993a722" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 35249 } } }
I20260806 21:16:24.324517 19976 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6e88d59cca4401792386acd201f9194 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:16:24.336369 19720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:24.324714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58222 (local address 127.16.178.62:46469)
0806 21:16:24.324867 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:24.324871 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:24.325878 (+ 1007us) server_negotiation.cc:407] Connection header received
0806 21:16:24.326070 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:24.326074 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:24.326132 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:24.326186 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:24.327101 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:24.327799 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:24.329705 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:24.330070 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:24.330138 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:24.332317 (+ 2179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:24.332322 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:24.332325 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:16:24.332367 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:24.334107 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:24.334714 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:24.334719 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:24.334720 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:24.334825 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:24.335214 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:24.335217 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:24.335219 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:24.335503 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:24.335610 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:24.335983 (+ 373us) server_negotiation.cc:299] Negotiation successful
0806 21:16:24.336158 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 21:16:24.336881 19946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:24.325353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59732 (local address 127.16.178.60:35249)
0806 21:16:24.325495 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:24.325500 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:24.325979 (+ 479us) server_negotiation.cc:407] Connection header received
0806 21:16:24.326195 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:24.326199 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:24.326253 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:24.326352 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:24.327497 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:24.328322 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:24.329668 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:24.330012 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:24.330111 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:24.332317 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:24.332322 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:24.332335 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:16:24.332367 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:24.334887 (+ 2520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:24.335356 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:24.335360 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:24.335361 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:24.335449 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:24.335771 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:24.335775 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:24.335776 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:24.336026 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:24.336170 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:24.336576 (+ 406us) server_negotiation.cc:299] Negotiation successful
0806 21:16:24.336724 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260806 21:16:24.342401 20028 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e4b6f9d7614a489599e60be14993a722: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260806 21:16:24.342564 19841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:24.316655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57260 (local address 127.16.178.61:40205)
0806 21:16:24.316803 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:24.316808 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:24.325211 (+ 8403us) server_negotiation.cc:407] Connection header received
0806 21:16:24.325352 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:24.325355 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:24.325410 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:24.325481 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:24.326491 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:24.327224 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:24.328418 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:24.328757 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:24.328835 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:24.338505 (+ 9670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 21:16:24.338510 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 21:16:24.338514 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 21:16:24.338553 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 21:16:24.340418 (+ 1865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:24.340989 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:24.340994 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:24.340996 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:24.341095 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 21:16:24.341458 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 21:16:24.341463 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 21:16:24.341465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 21:16:24.341720 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 21:16:24.341841 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:24.342229 (+ 388us) server_negotiation.cc:299] Negotiation successful
0806 21:16:24.342381 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260806 21:16:24.342587 20028 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 21:16:24.347922 19901 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e4b6f9d7614a489599e60be14993a722 [term 1 FOLLOWER]: Refusing update from remote peer efb0fb1afff74bbf86828a74fe217e2e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:16:24.348110 19792 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a6e88d59cca4401792386acd201f9194 [term 1 FOLLOWER]: Refusing update from remote peer efb0fb1afff74bbf86828a74fe217e2e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 21:16:24.349536 19953 consensus_queue.cc:1035] T 00000000000000000000000000000000 P efb0fb1afff74bbf86828a74fe217e2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4b6f9d7614a489599e60be14993a722" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 35249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:16:24.350214 19993 consensus_queue.cc:1035] T 00000000000000000000000000000000 P efb0fb1afff74bbf86828a74fe217e2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6e88d59cca4401792386acd201f9194" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 40205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 21:16:24.354492 20010 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4b6f9d7614a489599e60be14993a722 [sys.catalog]: SysCatalogTable state changed. Reason: New leader efb0fb1afff74bbf86828a74fe217e2e. Latest consensus state: current_term: 1 leader_uuid: "efb0fb1afff74bbf86828a74fe217e2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efb0fb1afff74bbf86828a74fe217e2e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 46469 } } peers { permanent_uuid: "a6e88d59cca4401792386acd201f9194" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 40205 } } peers { permanent_uuid: "e4b6f9d7614a489599e60be14993a722" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 35249 } } }
I20260806 21:16:24.354658 20010 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4b6f9d7614a489599e60be14993a722 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:16:24.355700 19976 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6e88d59cca4401792386acd201f9194 [sys.catalog]: SysCatalogTable state changed. Reason: New leader efb0fb1afff74bbf86828a74fe217e2e. Latest consensus state: current_term: 1 leader_uuid: "efb0fb1afff74bbf86828a74fe217e2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efb0fb1afff74bbf86828a74fe217e2e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 46469 } } peers { permanent_uuid: "a6e88d59cca4401792386acd201f9194" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 40205 } } peers { permanent_uuid: "e4b6f9d7614a489599e60be14993a722" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 35249 } } }
I20260806 21:16:24.355854 19976 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6e88d59cca4401792386acd201f9194 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:16:24.366483 19953 sys_catalog.cc:455] T 00000000000000000000000000000000 P efb0fb1afff74bbf86828a74fe217e2e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "efb0fb1afff74bbf86828a74fe217e2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efb0fb1afff74bbf86828a74fe217e2e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 46469 } } peers { permanent_uuid: "a6e88d59cca4401792386acd201f9194" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 40205 } } peers { permanent_uuid: "e4b6f9d7614a489599e60be14993a722" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 35249 } } }
I20260806 21:16:24.366644 19953 sys_catalog.cc:458] T 00000000000000000000000000000000 P efb0fb1afff74bbf86828a74fe217e2e [sys.catalog]: This master's current role is: LEADER
I20260806 21:16:24.366778 20006 mvcc.cc:205] Tried to move back new op lower bound from 7315664831880155136 to 7315664831736643584. Current Snapshot: MvccSnapshot[applied={T|T < 7315664831880155136}]
I20260806 21:16:24.368312 20010 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4b6f9d7614a489599e60be14993a722 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "efb0fb1afff74bbf86828a74fe217e2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efb0fb1afff74bbf86828a74fe217e2e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 46469 } } peers { permanent_uuid: "a6e88d59cca4401792386acd201f9194" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 40205 } } peers { permanent_uuid: "e4b6f9d7614a489599e60be14993a722" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 35249 } } }
I20260806 21:16:24.368458 20010 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4b6f9d7614a489599e60be14993a722 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:16:24.371994 19993 sys_catalog.cc:455] T 00000000000000000000000000000000 P efb0fb1afff74bbf86828a74fe217e2e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "efb0fb1afff74bbf86828a74fe217e2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efb0fb1afff74bbf86828a74fe217e2e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 46469 } } peers { permanent_uuid: "a6e88d59cca4401792386acd201f9194" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 40205 } } peers { permanent_uuid: "e4b6f9d7614a489599e60be14993a722" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 35249 } } }
I20260806 21:16:24.372167 19993 sys_catalog.cc:458] T 00000000000000000000000000000000 P efb0fb1afff74bbf86828a74fe217e2e [sys.catalog]: This master's current role is: LEADER
I20260806 21:16:24.373438 19976 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6e88d59cca4401792386acd201f9194 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "efb0fb1afff74bbf86828a74fe217e2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efb0fb1afff74bbf86828a74fe217e2e" member_type: VOTER last_known_addr { host: "127.16.178.62" port: 46469 } } peers { permanent_uuid: "a6e88d59cca4401792386acd201f9194" member_type: VOTER last_known_addr { host: "127.16.178.61" port: 40205 } } peers { permanent_uuid: "e4b6f9d7614a489599e60be14993a722" member_type: VOTER last_known_addr { host: "127.16.178.60" port: 35249 } } }
I20260806 21:16:24.373569 19976 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6e88d59cca4401792386acd201f9194 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 21:16:24.374040 20035 mvcc.cc:205] Tried to move back new op lower bound from 7315664831880155136 to 7315664831736643584. Current Snapshot: MvccSnapshot[applied={T|T < 7315664831880155136}]
I20260806 21:16:24.374106 20004 catalog_manager.cc:1347] Generated new cluster ID: 4eeeebda9d044bca9539fa0f46c68f87
I20260806 21:16:24.374163 20004 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 21:16:24.423439 20004 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 21:16:24.425037 20004 catalog_manager.cc:1504] Loading token signing keys...
I20260806 21:16:24.437229 20004 catalog_manager.cc:5868] T 00000000000000000000000000000000 P efb0fb1afff74bbf86828a74fe217e2e: Generated new TSK 0
I20260806 21:16:24.438256 20004 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 21:16:24.479548 19841 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:16:24.476378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57262 (local address 127.16.178.61:40205)
0806 21:16:24.476517 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:24.476522 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:24.478361 (+ 1839us) server_negotiation.cc:407] Connection header received
0806 21:16:24.478582 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:24.478586 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:24.478642 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:24.478696 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:24.479403 (+ 707us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57262: BlockingRecv error: recv got EOF from 127.0.0.1:57262 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260806 21:16:24.481132 19720 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:16:24.477167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58238 (local address 127.16.178.62:46469)
0806 21:16:24.477284 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:24.477288 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:24.480116 (+ 2828us) server_negotiation.cc:407] Connection header received
0806 21:16:24.480327 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:24.480331 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:24.480402 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:24.480459 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:24.481004 (+ 545us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58238: BlockingRecv error: recv got EOF from 127.0.0.1:58238 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260806 21:16:24.482636 19946 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:16:24.477839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59734 (local address 127.16.178.60:35249)
0806 21:16:24.477994 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:24.477998 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:24.481663 (+ 3665us) server_negotiation.cc:407] Connection header received
0806 21:16:24.481856 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:24.481859 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:24.481916 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:24.481969 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:24.482505 (+ 536us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59734: BlockingRecv error: recv got EOF from 127.0.0.1:59734 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260806 21:16:24.489240 19841 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:16:24.485952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57276 (local address 127.16.178.61:40205)
0806 21:16:24.486078 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:24.486083 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:24.487816 (+ 1733us) server_negotiation.cc:407] Connection header received
0806 21:16:24.488187 (+ 371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:24.488192 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:24.488312 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:24.488404 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:24.489112 (+ 708us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57276: BlockingRecv error: recv got EOF from 127.0.0.1:57276 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260806 21:16:24.495703 19841 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:16:24.493742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57278 (local address 127.16.178.61:40205)
0806 21:16:24.493869 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:24.493874 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:24.494561 (+ 687us) server_negotiation.cc:407] Connection header received
0806 21:16:24.494792 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:24.494796 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:24.494856 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:24.494913 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:24.495578 (+ 665us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57278: BlockingRecv error: recv got EOF from 127.0.0.1:57278 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260806 21:16:24.502290 19841 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:16:24.500521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57286 (local address 127.16.178.61:40205)
0806 21:16:24.500646 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:24.500650 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:24.501255 (+ 605us) server_negotiation.cc:407] Connection header received
0806 21:16:24.501495 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:24.501498 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:24.501554 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:24.501607 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:24.502169 (+ 562us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57286: BlockingRecv error: recv got EOF from 127.0.0.1:57286 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260806 21:16:24.517654 19841 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:16:24.515316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57298 (local address 127.16.178.61:40205)
0806 21:16:24.515522 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:24.515528 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:24.516407 (+ 879us) server_negotiation.cc:407] Connection header received
0806 21:16:24.516667 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:24.516671 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:24.516740 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:24.516816 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:24.517515 (+ 699us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57298: BlockingRecv error: recv got EOF from 127.0.0.1:57298 (error 108)
Metrics: {"server-negotiator.queue_time_us":106}
W20260806 21:16:24.530614 19841 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:16:24.528691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57302 (local address 127.16.178.61:40205)
0806 21:16:24.528839 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:24.528845 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:24.529514 (+ 669us) server_negotiation.cc:407] Connection header received
0806 21:16:24.529735 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:24.529739 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:24.529810 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:24.529866 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:24.530479 (+ 613us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57302: BlockingRecv error: recv got EOF from 127.0.0.1:57302 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260806 21:16:24.550733 19841 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:16:24.549418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57312 (local address 127.16.178.61:40205)
0806 21:16:24.549556 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:24.549561 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:24.549803 (+ 242us) server_negotiation.cc:407] Connection header received
0806 21:16:24.549953 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:24.549957 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:24.550012 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:24.550086 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:24.550605 (+ 519us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57312: BlockingRecv error: recv got EOF from 127.0.0.1:57312 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260806 21:16:24.586102 19841 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:16:24.584950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57324 (local address 127.16.178.61:40205)
0806 21:16:24.585133 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:24.585137 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:24.585233 (+ 96us) server_negotiation.cc:407] Connection header received
0806 21:16:24.585371 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:24.585374 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:24.585427 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:24.585501 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:24.585953 (+ 452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57324: BlockingRecv error: recv got EOF from 127.0.0.1:57324 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260806 21:16:24.653564 19841 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:16:24.652514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57332 (local address 127.16.178.61:40205)
0806 21:16:24.652691 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:24.652695 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:24.652765 (+ 70us) server_negotiation.cc:407] Connection header received
0806 21:16:24.652894 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:24.652897 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:24.652947 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:24.653019 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:24.653405 (+ 386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57332: BlockingRecv error: recv got EOF from 127.0.0.1:57332 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260806 21:16:24.785173 19841 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:16:24.783992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57348 (local address 127.16.178.61:40205)
0806 21:16:24.784181 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:24.784186 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:24.784290 (+ 104us) server_negotiation.cc:407] Connection header received
0806 21:16:24.784476 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:24.784478 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:24.784528 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:24.784595 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:24.785047 (+ 452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57348: BlockingRecv error: recv got EOF from 127.0.0.1:57348 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260806 21:16:25.044778 19841 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 21:16:25.043513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57352 (local address 127.16.178.61:40205)
0806 21:16:25.043691 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:25.043697 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:25.043809 (+ 112us) server_negotiation.cc:407] Connection header received
0806 21:16:25.044002 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:25.044004 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:25.044057 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:25.044151 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 21:16:25.044662 (+ 511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57352: BlockingRecv error: recv got EOF from 127.0.0.1:57352 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 21:16:25.312875 19992 catalog_manager.cc:1259] Loaded cluster ID: 4eeeebda9d044bca9539fa0f46c68f87
I20260806 21:16:25.312970 19992 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a6e88d59cca4401792386acd201f9194: loading cluster ID for follower catalog manager: success
I20260806 21:16:25.314790 19992 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a6e88d59cca4401792386acd201f9194: acquiring CA information for follower catalog manager: success
I20260806 21:16:25.315985 19992 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a6e88d59cca4401792386acd201f9194: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 21:16:25.344290 20028 catalog_manager.cc:1259] Loaded cluster ID: 4eeeebda9d044bca9539fa0f46c68f87
I20260806 21:16:25.344372 20028 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e4b6f9d7614a489599e60be14993a722: loading cluster ID for follower catalog manager: success
I20260806 21:16:25.346007 20028 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e4b6f9d7614a489599e60be14993a722: acquiring CA information for follower catalog manager: success
I20260806 21:16:25.347182 20028 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e4b6f9d7614a489599e60be14993a722: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 21:16:25.562897 20135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:25.559180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57362 (local address 127.16.178.61:40205)
0806 21:16:25.559439 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:25.559443 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:25.559469 (+ 26us) server_negotiation.cc:407] Connection header received
0806 21:16:25.559605 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:25.559607 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:25.559741 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:25.559855 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:16:25.560093 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:25.560671 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:25.561527 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:25.561749 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:25.561822 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:25.562096 (+ 274us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:16:25.562579 (+ 483us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:16:25.562675 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:25.562704 (+ 29us) server_negotiation.cc:299] Negotiation successful
0806 21:16:25.562760 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":81,"threads_started":1}
I20260806 21:16:25.569518 20143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:25.566004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59738 (local address 127.16.178.60:35249)
0806 21:16:25.566260 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:25.566264 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:25.566278 (+ 14us) server_negotiation.cc:407] Connection header received
0806 21:16:25.566360 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:25.566362 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:25.566505 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:25.566605 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:16:25.566775 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:25.567304 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:25.568188 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:25.568414 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:25.568504 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:25.568745 (+ 241us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:16:25.569195 (+ 450us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:16:25.569285 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:25.569310 (+ 25us) server_negotiation.cc:299] Negotiation successful
0806 21:16:25.569360 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":86,"threads_started":1}
I20260806 21:16:25.576105 20135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:25.572382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57378 (local address 127.16.178.61:40205)
0806 21:16:25.572509 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:25.572513 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:25.572920 (+ 407us) server_negotiation.cc:407] Connection header received
0806 21:16:25.573095 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:25.573098 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:25.573155 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:25.573225 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:16:25.573472 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:25.574224 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:25.575123 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:25.575336 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:25.575422 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:25.575661 (+ 239us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:16:25.575731 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:16:25.575788 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:25.575880 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 21:16:25.575932 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 21:16:25.576848 20143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:25.572834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59746 (local address 127.16.178.60:35249)
0806 21:16:25.572945 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:25.572948 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:25.572960 (+ 12us) server_negotiation.cc:407] Connection header received
0806 21:16:25.573012 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:25.573014 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:25.573062 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:25.573122 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:16:25.573384 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:25.574155 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:25.575487 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:25.575820 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:25.575892 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:25.576383 (+ 491us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:16:25.576470 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:16:25.576516 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:25.576679 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 21:16:25.576732 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 21:16:25.577351 20158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 21:16:25.572362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58240 (local address 127.16.178.62:46469)
0806 21:16:25.572619 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0806 21:16:25.572623 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 21:16:25.572903 (+ 280us) server_negotiation.cc:407] Connection header received
0806 21:16:25.573010 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 21:16:25.573014 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 21:16:25.573174 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 21:16:25.573283 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 21:16:25.573593 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:25.574104 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:25.575487 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 21:16:25.575832 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 21:16:25.575930 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 21:16:25.576438 (+ 508us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 21:16:25.576785 (+ 347us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 21:16:25.576865 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 21:16:25.577155 (+ 290us) server_negotiation.cc:299] Negotiation successful
0806 21:16:25.577214 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":64,"threads_started":1}
I20260806 21:16:25.585764 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 19621
I20260806 21:16:25.604143 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 19728
I20260806 21:16:25.617045 17096 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ1T3_n/build/debug/bin/kudu with pid 19844
2026-08-06T21:16:25Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3287 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6584 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (298778 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST